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AFOTEC’S  EMBLEM 


Shown  on  the  front  cover,  AFOTEC’s  official  emblem  was  designed  in 
1974  based  upon  the  new  Center’s  responsibilities  and  the  roles  and  missions 
of  the  United  States  Air  Force  as  outlined  in  the  contemporary  edition  of  AFM 
1-1.  The  emblem’s  heraldry  features  four  blue  and  gold  "deltoids."  They 
appear  as  a  flight  of  delta  wing  aircraft  leaving  white  contrails  against  a  blue 
sky.  Each  of  the  deltoids  has  a  symbolic  meaning,  representing  four  funda¬ 
mental  military  objectives  of  the  United  States:  to  deter  aggression,  to  resolve 
conflicts  on  favorable  terms,  to  achieve  national  objectives,  and  to  promote  a 
secure  international  environment.  The  blue  and  gold  deltoid  color  scheme 
subdivides  these  four  fundamental  objectives  into  eight  of  the  specified 
missions  and  tasks  of  the  United  States  Air  Force:  (1)  strategic  aerospace,  (2) 
coimterair,  (3)  air  interdiction,  (4)  dose  air  support,  (5)  aerospace  defense,  (6) 
reconnaissance,  (7)  electronic  warfare,  and  (8)  airlift.  The  white  contrails 
signify  the  test  and  evaluation  process,  which  follows  the  concept  formulation, 
validation,  and  development  of  systems  and  equipment.  The  red  scales  in  the 
foreground  portray  AFOTEC’s  impartial  and  independent  assessment  of  system 
performance  as  weighed  against  the  Air  Force’s  tasks  and  missions. 

To  represent  the  50  years  of  Air  Force  operational  testing  since  the 
formation  of  AFOTEC’s  predecessor,  the  Air  Proving  Groimd  Command,  the 
cover  shows  head-on  drawings  of  two  aircraft:  the  B-29,  which  was  the  most 
advanced  bomber  of  World  War  11;  and  today’s  B-2,  which  will  help  give  the 
Air  Force  global  reach  and  global  power  into  the  2l8t  century. 
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PREFACE 


This  study  is  primarily  intended  as  background  information  for  people 
involved  with  operational  test  and  evaluation  (OT&E),  especially  as  conducted 
by  the  Air  Force.  Presented  from  the  perspective  of  frie  Air  Force  Operational 
Test  and  Evaliiation  Center,  it  expands  on  a  previous  publication  titled  "An 
Introduction  to  Operational  Testing  and  AFOTEC"  (last  updated  in  July  1991), 
which  recent  events  have  rendered  obsolete.  As  described  herein,  operational 
testing  has  been  evolving  since  World  War  11.  Understanding  the  policies  and 
organizational  features  of  today^s  OT&E  requires  some  knowledge  of  this 
history. 

The  undersigned  completed  this  publication  on  the  eve  of  departing  for 
a  new  assignment  after  six  rewarding  years  at  AFOTEC.  This  period, 
espedaUy  the  last  year,  has  been  marked  by  frmdamental  changes  in  the  Air 
Force's  OT&E  mission.  Although  a  slower  pace  of  diange  in  the  coming  years 
may  allow  this  study  to  remain  useful  for  some  time  as  a  current  reference  as 
well  as  a  historical  document,  a  revision  will  eventually  become  necessary. 
With  this  in  mind,  please  forward  any  corrections,  suggestions,  updates,  or 
additional  information  to  HQ  AFOTEC/RS,  8500  Gibson  Blvd  SE,  Kirtland 
AFB  NM  87117-5578  (DSN  246-5341). 


LAWRENCE  R.  BENSON 
Director  of  Research  Services 
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HISTORY  OF 

Am  FORCE  OPERATIONAL  TEST  AND  EVALUATION  (OT&E): 
MISSION,  ORGANIZATION,  AND  POLICY 


The  benefits  of  testing  weapons  under  realistic  conditions  before 
using  them  in  combat  became  apparent  during  World  War  11.  But  how  and 
when  these  tests  should  be  conducted  and  who  should  do  them  were  questions 
that  the  Air  Force  answered  in  different  ways  over  the  next  50  years.  The 
evolution  of  the  Air  Force  Operational  Test  and  Evaluation  Center  (AFOTEC) 
firom  a  small  management  headquarters  in  1974  to  the  Air  Force’s  central 
operational  test  agency  in  1992  represents  the  latest  chapter  in  this  quest. 
Before  recounting  the  history  of  operational  testing,  however,  an  introduction 
to  today’s  testing  process  and  related  activities  as  they  are  presently  defined 
will  help  put  past  developments  in  a  clearer  perspective. 

The  OT&E  Process 


Definitions 

Within  the  Department  of  Defense  (DoD),  test  and  evaluation  (T&E) 
encompasses  a  wide  range  of  activities,  broadly  categorized  as  development  test 
and  evaluation  G3T&E)  and  operational  test  and  evaluation  (OT&E).  Although 
either  type  of  T&E  may  occur  at  any  point  in  the  life  pyde  of  a  system,  DT&E 
usiially  begins  first  It  focuses  on  engineering  analyses,  design  alternatives, 
performance  measurements,  and  compliance  with  contract  specifications  in  a 
controlled  environment  DT&E  includes  both  contractor  and  government- 
conducted  projects,  ranging  fiium  laboratory  ezperimentatioh  to  flight  testing. 
Contractor  testing  includes  preproduction  qualification  testing  (PPQT)  and 
production  qualification  testing  (PQT).  OT&E  traditionally  starts  later  than 
DT&E  and  focuses  on  the  overall  performance  of  a  system  in  its  intended 
operational  environment.  In  view  of  OT&E’s  role  in  the  acquisition  process, 
DoD  and  the  services  have  assigned  responsibility  for  it  to  independent 
operational  test  agencies  (OTAs),  such  as  /^OTEC. 

OT&E.  By  law  (Title  10  of  the  US  Code)  operational  test  and 
evaluation  is  defined  as  "the  field  test,  under  realistic  combat  conditions,  of 
any  item  of  (or  key  component  of)  weapons,  equipment,  or  munitions  for  the 
purpose  of  detennining  the  effectiveness  and  suitability  [of  these  items]  for  use 
in  combat  by  t3rpical  militaxy  users;  and  the  evaluation  of  the  results  of  such 
test"'  Operational  effectiveness  primarily  concerns  how  systems  perform 
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when  employed,  while  operational  suitability  involves  how  well  they  can  be 
kept  available  for  use  (for  example,  reliability,  maintainability,  and  supporta- 
bility).  Operational  (or  "user")  requirements,  drawn  up  by  the  operating 
commands,  provide  the  main  criteria  against  which  to  measure  the  system’s 
performance.  The  findings  of  OT&E  contribute  to  decisions  on  the  acquisition 
of  new  systems,  improvements  to  systems  already  being  produced,  modifica¬ 
tions  of  systems  deployed  in  the  field,  and  other  aspects  of  their  operational 
capabilities.  OT&E  has  several  sub-categories  and  related  activities  based 
largely  on  the  maturity  of  the  system  being  examined. 

lOT&E.  Production  or  production-representative  hardware  funded 
by  research,  development,  test  and  evaluation  (RDT&E)  appropriations  receive 
initial  OT&E  (lOT&E).  The  most  common  purpose  of  lOT&E  is  to  assist  in 
making  decisions  in  the  acquisition  process,  especially  for  beyond  low  rate 
initial  production  (B-LRIP)  or  full  rate  production  at  Milestone  in  (see  Figure 
1).  For  low  volume  or  one-of-a-kind  systems,  such  as  ships,  satellites, 
simulators,  and  command  centers,  lOT&E  is  also  used  to  support  turnover 
decisions  firom  the  developing  to  the  operating  command.  Within  the  Air 
Force,  lOT&E  has  normally  overlapped  with  DT&E,  sharing  many  of  the  same 
resources. 

FOT&E.  After  a  system  enters  production,  it  may  undergo  follow-on 
OT&E  (FOT&E).  FOT&E  refines  lOT&E  estimates,  assessing  the  capability 
of  the  fuU  system  and  any  modifications,  and  reevaluates  the  system  against 
changing  operational  needs.  AFOTEC  conducts  some  Air  Force  FOT&Es  for 
high-cost,  high-risk,  or  potentially  controversial  programs.  The  operating 
commands,  which  conduct  most  FOT&Es,  are  also  responsible  for  tactics 
development  and  evaluation  (TD&E),  confirming  training  requirements,  and 
refining  doctrine  for  weapons  employment. 

QOT&E.  Systems  that  do  not  require  R&D  funding  receive  qualifica¬ 
tion  OT&E  (QOT&E).  These  include  existing  systems  given  a  new  mission  or 
modified  to  improve  capabilities  as  well  as  contractor  ofT-the-shelf  (COTS)  or 
nondevelopmental  items  (NDI).  Procedures  followed  for  QOT&E  are  similar 
to  those  used  for  lOT&E,  and  the  actual  testing  may  be  combined  with 
Qualification  Test  and  Evaluation  (QT&E),  which  resembles  DT&E. 

OAs.  To  help  reduce  risk  in  early  acquisition  decisions,  DoD  has 
called  upon  its  operational  test  agencies  to  examine  new  systems  before  there 
is  production-representative  hardware  to  test.  When  this  activity  occurs 
during  the  demonstration  and  validation  stage  or  before,  it  is  known  as  an 
early  operational  assessment  (EOA).  When  it  supports  another  decision  prior 
to  production  approval  (Milestone  IID»  it  is  called  an  operational  assessment 
(OA).  OAs  may  use  technology  demonstratora,  prototypes,  mockups,  engineer- 
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ing  development  models,  or  simulations.  The  objectives  of  these  assessments 
are  to  improve  planning  by  highlighting  program  documentation  status  and  the 
readiness  of  a  system  for  lOT&E,  examine  the  significance  of  programmatic 
voids  and  early  testing  trends,  and  conduct  special  field  tests  or  simulations 
as  directed.  The  four  services  defined  the  OA  and  EOA  in  1989. 

Multiservice  T&E/OT&E.  When  two  or  more  services  or  federal 
agencies  test  a  system  funded  with  KDT&E  money,  it  is  considered  a 
multiservice  test  and  evaluation.  In  these  cases,  DoD  appoints  a  lead  service, 
which  conducts  the  test  according  to  its  regulations.  For  multiservice  OT&E, 
the  fo\u:  OTAs  follow  procedures  and  arrangements  outlined  in  an  umbrella 
memorandum  of  agreement. 

JT&E.  Joint  test  and  evaluation  (JT&E)  programs  also  involve  two 
or  more  of  the  services,  but  as  a  rule  are  administered  and  largely  funded  by 
the  Office  of  the  Secretary  of  Defense  (OSD)  as  a  separate  category  of  testing 
that  does  not  represent  acquisition  efforts.  Conducted  by  specially  formed  joint 
test  forces,  JT&E  may  be  either  developmental  or  operational  in  emphasis.^ 
During  the  19708,  many  JT&E  programs  were  chartered  as  large  and  "open- 
ended"  efforts,  but  in  1986  OSD  began  limiting  their  scope  and  setting  a  goal 
of  two  or  three  years  for  their  execution.^ 

OUEs  and  Demonstrations.  Although  not  formally  defined, 
operational  utility  evaluations  (OUEs)  have  been  periodically  conducted  to 
meet  a  specific  purpose  directed  by  higher  headquarters.  They  normally  assess 
how  well  a  future  system  would  satisfy  overall  user  requirements  if  it  performs 
as  advertised.  Because  of  their  early  timing  in  the  acquisition  process,  OUEs 
can  rely  heavily  on  modeling  and  simulation.  On  other  occasions  when 
hardware  is  available,  operational  test  agendes  may  conduct  restricted  tests, 
characterized  as  demonstrations,  to  show  how  the  existing  equipment  performs 
certain  functions.^ 

Program  Milestones 

Most  of  the  activities  defined  above  support  defense  acquisition 
dedsions.  The  structure  of  the  acquisition  process,  which  has  changed 
periodically  over  the  years,  underwent  a  migor  revision  with  the  new  DoD 
"5000"  series  publications  signed  on  23  February  1991.  Figure  1  on  the  next 
page  shows  the  phases  and  milestones  (dedsion  points)  in  the  acqmsition  (yde, 
with  accompanying  T&E  activities  listed  beneath.* 

The  Defense  Acquisition  Board  (DAB)  reviews  major  programs,  i.e., 
Acquisition  Category  (ACAT)  I  and  U,  at  each  milestone.  After  being 
streamlined  in  1990,  members  of  the  DAB  induded  the  Undersecretary  of 
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Defense  for  Acquisition  (USD(A)),  Vice  Chairman  of  the  Joint  Chiefs  of  Staff 
(JCS),  OSD’s  Director  of  OT&E  (DOT&E),  Director  of  Defense  Research  and 
Engineering  (DDR&E),  Assistant  Secretazy  for  Program  Analysis  and  Evalua¬ 
tion  (PA&E),  DoD  Comptroller,  and  the  three  service  acquisition  executives 
(SAEs).  Its  supporting  structure  includes  committees  for  conventional, 
strategic,  and  command,  control,  communications,  and  intelligence  (C^I) 
systems.  Service  bodies,  such  as  the  Air  Force  Systems  Acquisition  Review 
Council  (AFSARC),  follow  milestones  similar  to  the  DAB  for  those  programs 
over  which  they  have  authority.  The  services  may  hirther  delegate  decisions 
for  smaller  programs  (ACAT  III  and  IV)  to  subordinate  commands.  Today, 
OT&E  is  an  integral  part  of  the  acquisition  process.  This  was  not  always  the 
case  in  the  past. 
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Figure  1.  Acquisitioii  Milestones  and  OT&E  Activities 


Air  Force  OT&E  Before  AFQTEC 

How  did  operational  testing  become  an  influential  part  of  defense 
acquisition?*  In  essence,  what  is  now  Air  Force  T&E  dates  back  to  1909, 
when  the  Chief  of  the  Signal  Cozps  issued  the  following  "test  directive"  to  ILt 
Bexyamin  Foulois  after  the  Wright  Brothers  delivered  an  early  flying  machine 
to  the  Army.  "Your  orders  are  simple,  lieutenant.  You  are  to  evaluate  the  air¬ 
plane.  Just  take  plenty  of  spare  parts~and  teach  yourself  to  fly."*  Foulois 
soon  improved  the  operational  effectiveness  of  the  aircraft  by  adding  a  seat, 
safety  belt,  and  lancUng  gear. 
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From  World  War  I  to  World  War  II 

By  the  time  the  United  States  entered  World  War  I  in  1917,  the 
European  powers-testing  their  aircraft  over  the  trenches  for  almost  three 
years**were  far  ahead  in  all  aspects  of  military  aviation.  American  airmen  had 
to  fly  British  and  French  planes  into  combat. 

During  the  1920s  and  1930s,  despite  tight  budgets,  the  Army  Air 
Corps  established  and  built  up  the  Materiel  Division,  a  centralized  organiza¬ 
tion  that  combined  RDT&E-type  functions  with  logistics  at  Wright  and 
Patterson  Fields  near  Dayton,  OH.  Reflecting  peacetime  priorities  and  the 
revolutionary  strides  being  made  in  aviation  technology  at  the  time,  the 
Materiel  Division’s  testing  emphasized  basic  engineering,  contract  compliance, 
and  flying  characteristics  rather  than  wartime  operations.*  With  the  coming 
of  World  War  n  in  Europe,  however,  the  gradually  emerging  concept  of  a 
proving  grotmd  to  more  realistically  test  air  weapons  became  reality. 

Air  Proving  Ground,  1941-1957 

Six  months  before  Pearl  Harbor,  in  May  1941,  the  Army  Air  Forces 
(AAF)  created  the  Air  Proving  Ground  at  Eglin  Field,  FL.  Upgraded  to  the  Air 
Proving  Ground  Command  (APGC)  in  April  1942,  its  mission  rapidly  expanded 
fiiom  "service  tests"  of  ordnance  and  aviation  equipment  to  testing  new  aircraft 
for  operational  fitness  and  tactical  suitability.  Brig  Gen  Grandison  Gardner, 
a  protege  of  AAF  Commanding  General  Hap  Arnold,  served  as  the  APGC’s 
commander  throughout  the  war.  Like  AFOTEC  today,  the  Air  Proving  Ground 
Command  was  independent  of  developing  and  using  commands  and  reported 
directly  to  General  Arnold  or  his  Army  Air  Forces  Board.  By  war’s  end,  the 
Command  had  conducted  more  than  2,800  tests  on  a  wide  variety  of  equip¬ 
ment,  including  almost  all  of  the  notable  American  fighters  and  bombers  of 
World  War  H.  In  addition  to  perfecting  the  use  of  proximity  fuzes,  napalm, 
incendiary  bombs,  and  fighter  escorts,  some  credit  it  with  originating  the  role 
of  the  fighter-bomber  as  a  key  element  in  tactical  air  power.  The  Proving 
Ground  even  ushered  in  the  American  employment  of  cruise  missiles  by  testing 
and  deploying  an  American  copy  of  the  German  V-1.* 

After  a  brief  loss  of  status  during  the  post-war  demobilization,  the 
APGC  was  recreated  as  one  of  the  early  mqjor  commands  (MAJCOMs)  of  the 
new  United  States  Air  Force  (USAF)  in  1948.  "In  determining  the  operational 
suitability  of  materiel  and  equipment,"  explained  its  first  post-war  commander, 
"every  effort  is  expended  to  simulate  combat  conditions."^*  The  outbreak  of 
the  Korean  War  in  1950  reawakened  interest  in  combat  readiness  throughout 
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the  Air  Force.  A  later  article  asserted  that  "APGC  is  a  major  command  for  one 
good  reason->to  assure  its  objectivity.  Responsible  to  no  one  but  the  Air  Force 
Chief  of  Staff,  Proving  Ground  calls  the  results  as  it  sees  them."^^  At  the  end 
of  the  Korean  War,  the  APGC  commander  explained  his  command’s  philosophy 
as  follows.  "We  don’t  concern  ourselves  about  the  factors  that  went  into  design 
of  the  equipment,  though  we  appreciate  that  they  must  be  niimerous  and 
complex....  We  try  to  find  out  only  one  thing-will  it  do  the  job?"^^ 

During  the  1950s,  APGC  grew  to  include  12,000  personnel  and  its 
own  "Air  Force  in  miniature"  consisting  of  almost  every  aircraft  and  missile 
then  in  the  inventory.  Its  primary  mission  was  conducting  "operational 
suitability  tests,"  defined  as  "tests  of  tactically  equipped  aircraft  materiel  or 
equipment  with  their  component  systems  and  support  items  to  determine  their 
effectiveness  in  combat."  These  tests  also  were  "designed  to  develop  improved 
operational  tactics  and  techniques....and  refine  support  and  training  require¬ 
ments."  In  addition  to  performing  operational  suitability  tests,  APGC  tested 
new  weapons  under  extreme  climactic  conditions  and  examined  their  reliability 
and  producability.  With  all  of  its  weapons  and  resources,  APGC  was  also 
called  upon  to  stage  impressive  firepower  demonstrations  as  part  of  Air  Force 
and  DoD  public  relations  programs.^ 

Priority  given  to  nuclear  capabilities  after  the  Korean  War  led  to 
deemphasis  of  conventional  weapons  and  traditional  combat  roles.  The  Air 
Proving  Ground  Command,  whidi  was  not  universally  popular  with  other 
elements  of  the  Air  Force,  appeared  to  have  grown  too  big  and  independent  for 
its  own  good.  Facing  large  budget  cuts  in  conventional  forces  in  1957,  the  Air 
Force  abolished  APGC  as  a  MAJCOM  and  merged  what  was  left  into  a  test 
center  of  the  Air  Research  and  Development  Command  (ARDC).^^ 

Why  did  the  Air  Force  do  away  with  its  first  independent  operational 
test  agency?  The  official  answer  was  "the  changing  technology  of  war"-with 
missiles  replacing  bullets-and  the  need  for  more  timely  production  appeared 
to  make  operational  testing  obsolete.  As  explained  by  the  APGC’s  last 
commander,  "once  production  starts,  it  is  costly  to  slow  down  or  hold  back."^‘ 
Part  of  the  underlying  reason  for  the  demise  of  the  Air  Proving 
Ground  Command  may  be  fotmd  by  looking  at  the  multi-phase  T&E  process 
of  the  time.^*  The  first  six  of  these  phs  emphasized  engineering  factors, 
contract  compliance,  and  the  functioning  of  equipment.  Not  until  operational 
suitability  testing  (Phase  VII)  did  APGrC  get  a  chance  to  see  how  a  weapon 
worked  under  operational  conditions.  In  view  of  this,  Gen  Hoyt  S.  Vandenberg, 
Air  Force  Chief  of  Staff  fix>m  1948-1953,  had  ordered  that  early  production 
schedules  be  slowed  down  to  give  APGC  time  to  complete  its  testing  and  make 
recommendations  for  design  changes  before  the  need  for  costly  retrofits.^^ 
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The  growing  Cold  War  technology  race,  however,  put  priority  on 
getting  complex  new  equipment  into  the  field  as  soon  as  possible.  This  put 
more  pressure  on  ARDC  not  to  slow  programs  down.  In  1956  the  Air  Force 
added  Phase  Vm,  "unit  operational  employment  testing,"  conducted  by  the 
using  commands.^”  By  the  time  APGG  and  the  MAJCOMs  had  a  chance  to 
assess  new  systems,  an  increasing  number  were  already  in  production. 
Although  operational  testing  could  still  help  in  the  system’s  employment  and 
support  as  well  as  document  requirements  for  future  modifications,  it  seemed 
to  contribute  relatively  little  to  the  ever  accelerating  acquisition  process.^ 

Streamlined  T&E,  1958*1964 

To  shorten  the  time  needed  to  get  ever  more  sophisticated  equipment 
firom  design  to  deployment,  the  Air  Research  and  Development  Command  had 
devised  the  concept  of  the  "weapon-system."  Initially  applied  to  Interconti¬ 
nental  Ballistic  Missiles  (ICBMs),  the  resulting  "total  package  procurement" 
policy  included  having  the  Air  Force  enter  into  a  partnership  relation  with 
contractors  to  eliminate  duplication.  Reflecting  this  philosophy,  ARDC  was 
expanded  to  create  Air  Force  Systems  Command  (AFSC)  in  1961.  For 
maximum  efddency,  the  new  management  approach  featured  a  high  degree  of 
"concurrency"  in  development,  testing,  production,  fhdUties  construction,  and 
other  support  preparations.”  The  coi^ensed  T&E  schedule  and  the  Air 
Force’s  growing  emphasis  on  missiles  and  nuclear  weapons,  even  in  the  tactical 
air  forces,  called  into  question  the  validity  of  traditional  operational  testing 
and  its  "fly-before-buy"  philosophy. 

In  conjimction  with  APGC’s  demise,  the  Air  Force  streamlined  the 
T&E  process  firom  the  eight  phases  of  testing  to  only  three  categories.  Cate¬ 
gories  I  and  n,  performed  respectively  by  the  contractor  and  AFSC’s  test 
centers,  were  equivalent  to  today’s  DT&E.  Category  IQ,  performed  by  the 
MAJCOMs,  was  similar  to  today’s  FOT&E.  Overall,  Air  Force  T&E  was  the 
most  formal  and  structured  of  the  four  services’  programs,  but  there  was  no 
longer  an3dhing  equivalent  to  today’s  lOT&E  before  production. 

The  Air  Force  OT&E  function  had  also  become  procedural  rather 
than  organizational.  Even  the  office  on  the  Air  Staff  that  had  interfaced  with 
APGC  since  1948  was  abolished,  so  there  was  no  central  operational  testing 
advocate  to  maintain  a  corporate  memory  or  to  balance  the  interests  of  the 
developer  and  user.  Within  the  scope  of  this  decentralized  system,  the 
MAJCOMs  did  perform  some  ^highly  professional  OT&Es.  These  tests, 
however,  still  usually  occurred  far  too  late  to  affect  equipment  design.  Instead, 
the  implementation  of  OT&E  findings  often  required  costly  retrofits,  modifica- 
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tions,  and  various  "workarounds"  in  the  field.  The  high  degree  of  concurrency- 
which  had  worked  so  well  in  the  crash  development  of  ICBMs-proved  less 
successful  when  applied  to  systems  with  more  flexible  missions,  such  as 
aircraft.*^ 

Growing  concern  with  this  situation  led  the  Air  Staff  to  form  a  Wea¬ 
pons  Effectiveness  Testing  Task  Force  in  December  1964.  After  highlighting 
the  la(^  of  valid  data  with  which  to  evaluate  weapons  effectiveness,  this 
interim  body  was  replaced  by  a  new  deputy  to  the  Deputy  Chief  of  Staff  (DCS) 
for  Plans  and  Operations  (AF/XOPW)  in  1965.  This  Deputy  Directorate  for 
OT&E  grew  to  over  75  people  who  monitored  test  programs,  coordinated  on 
sponsored  test  resources,  and  provided  the  Air  Force  with  centralized 
operational  testing  guidance  for  the  next  decade.^  Its  Air  Staff  organization¬ 
al  symbol  changed  to  AF/XOW  in  1967  and  AF/XOOW  in  1970.  After  being 
headed  by  brigadier  generals  in  1965  and  1966,  the  directorate  was  thereafter 
run  by  colonels;  however,  in  December  1970  the  Deputy  Director  of  Operations 
(normally  a  2-star  general)  was  assigned  an  additional  duty  as  the  DCS/Plans 
and  Operation’s  Assistant  for  OT&E.^ 

New  Focus  ou  Operational  Testing,  1965-73 

Combat  Evaluations.  The  war  in  Vietnam  presented  American 
weapons  with  unexpected  challenges,  ranging  firom  primitive  anti-guerilla 
tactics  to  rapidly  evolving  electronic  warfare  capabilities.  The  hot  and  humid 
jungle  environment  also  took  its  toll.  Problems  in  employment  and  mainten¬ 
ance,  which  might  have  been  corrected  with  earlier  OT&E,  had  to  be  dealt  with 
at  great  cost  in  the  field.  For  air-to-air  weapons,  the  Pacific  Air  Forces 
(PACAF)  conducted  "Combat  Sage,"  as  its  contribution  to  the  existing  Weapon 
System  Evaluation  Program  (WSEP).^  Wartime  requirements  for  new 
systems  also  led  to  new,  high  priority  development  and  testing  programs,  such 
as  those  for  fixed  wing  gunships  and  electronic  countermeasures.^  Combat 
evaluations,  such  as  the  highly  publicized  Combat  Lancer  deployment  of  F- 
lllAs  to  Thailand  in  1968  (when  three  of  six  new  aircraft  were  lost  in  the  first 
five  weeks),  sometimes  proved  risky.” 

Southeast  Asia  Performance  Deficiencies.  Problems  affected  all 
the  armed  services.  In  a  sample  of  22  weapon  systems  deployed  to  Southeast 
Asia  firom  1965-1970,  Department  of  Defense  studies  found  all  but  one  had 
suffered  meqor  deficiencies  in  the  field.  Some  placed  blame  on  the  fact  that 
only  three  of  these  weapons  had  undergone  OT&E  prior  to  production  deci¬ 
sions.”  As  the  result  of  such  findings  and  other  embarrassing  problems  with 
performance  and  cost  overruns  on  new  systems,  the  potential  benefits  of  OT&E 
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became  the  focus  of  much  attention.  The  acquisition  of  fewer  but  more 
complex  and  expensive  systems  with  longer  service  lives  was  allowing  less 
room  for  mistakes  than  in  the  past. 

Ideas  for  Change.  Starting  in  1969  with  reports  by  task  forces  of 
the  President’s  Science  Advisory  Committee  and  the  Defense  Sdence  Board, 
DoD  began  looking  at  how  OT&E  could  better  contribute  to  procurement 
dedsions-which  now  came  under  the  purview  of  the  Defense  Systems 
Acquisition  Review  Council  (DSARC)-forerunner  of  today’s  DAB.  There  was 
a  growing  perception  within  DoD  and  Congress  that  both  the  developing  and 
operating  commands  had  too  much  of  a  stake  in  the  success  of  major  acquisi¬ 
tion  programs  to  be  trusted  with  doing  aU  of  the  testing  and  subsequent 
evaluations. 

Calls  for  Independent  OT&E.  The  pressure  on  the  services  to 
reorganize  their  OT&E  function  gained  momentum  in  the  early  1970s.  After 
a  detailed  study  of  operational  testing,  a  Presidential  Blue  Ribbon  Panel 
recommended  on  1  July  1970  that  thei^  be  an  OT&E  organization,  inde¬ 
pendent  of  the  developer  and  user,  reporting  directly  to  the  chief  of  each 
service.^  Deputy  Secretaiy  of  Defense  David  Packard  repeated  this  recom¬ 
mendation  in  a  memorandum  to  the  services  on  11  February  1971,  and  on  21 
April  1971,  he  introduced  the  definition  of  "initial  operational  test  and  evalua¬ 
tion"  (lOT&E)  of  new  systems  to  assist  in  acquisition  decisions.”  Some  key 
members  of  Congress  supported  Packaid’s  ideas  on  both  the  independence  and 
timing  of  operational  testing,  and  Public  Law  92-156  of  17  November  1971 
required  submission  of  OT&E  data  before  weapons  procurement,”  In  Decem¬ 
ber  1972  the  Commis.sion  on  Government  Procurement  recommended  that 
OT&E  start  as  early  as  possible  in  major  system  acquisitions  and  be  conducted 
by  an  activity  separate  from  developer  and  user  organizations.’*  As  described 
later  (pages  23-25),  the  Army  and  Navy  quickly  complied  by  designating 
independent  OTAs. 

Initial  Air  Force  Response.  The  Air  Force  did  not  respond 
promptly  to  these  calls  and  create  a  separate  operational  test  agency  in  the 
field.  Afier  the  Bolender  study  (named  for  the  brigadier  general  who  headed 
it),  whidi  was  completed  in  September  1970,  the  Air  Force  did  initiate  several 
management  improvements.”  These  included  designation  of  the  2-star 
position  of  Assistant  for  OT&E,  who  could  report  directly  to  the  CSAF.”  It 
also  restructured  T&E  firom  the  former  system  of  category  testing  to  the  basic 
dichotomy  of  DT&E  and  OT&E  still  in  effect  today,  with  the  using  commands 
responsible  for  the  latter.”  The  Air  Staff  did  not  conciu:  with  the  Procure¬ 
ment  Commission’s  recommendation  to  create  an  independent  OTA,  contending 
that  existing  checks  and  balances  were  sufficient  to  prevent  bias  in  test 
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reporting.  In  May  1973  the  Vice  CSAF  rejected  a  request  from  the  Deputy 
Director  of  Defense  Research  and  Engineering  for  Test  and  Evaluation 
(DDDR&E(T&E))  to  reconsider  the  Air  Force  position.^  Meanwhile,  the  Air 
Force’s  delay  in  establishing  an  operational  test  agenpy  had  been  criticized  by 
the  General  Accoimting  Office  (GAO)  in  a  statement  by  the  Comptroller 
General  on  26  March  1973.^ 

Decision  to  Create  AFTEC  While  Retaining  MAJCOM  OT&E 

The  growing  pressures  from  OSD  and  Congress  eventually  overcame 
the  Air  Force’s  reluctance  to  comply  with  the  Commission’s  recommendation. 
In  late  September  Gen  George  S.  Brown,  CSAF  (and  former  commander  of 
AFSC)  ordered  the  Air  Staff  to  start  planning  for  a  new  OTA,  and  one  month 
later  he  advised  DDDR&E(T&E)  of  this  plan.”  After  staffing  the  proposal 
and  formally  notifying  the  MAJCOMs  on  24  November,”  Headquarters  Air 
Force  issued  the  official  directive  for  the  establishment  of  the  Air  Force  Test 
and  Evaluation  Center  (AFTEC)  on  11  December.*  To  help  assure  "complete 
objectivity,”  AFTEC  was  to  be  "independent  of  those  Air  Force  commands 
which  develop,  procure,  and  use  Air  Force  weapons  and  subsystems."”  In 
deference  to  the  Air  Force,  Department  of  Defense  Directive  (DoDD)  5000.3, 
"Test  and  Evaluation,"  had  allowed  for  a  "limited  number  ofl..m£gor  field 
agencies"  for  OT&E  when  first  published  in  January  1973,  but  a  change  in 
early  1974  strudc  out  this  exemption.  The  directive  then  read  that  "in  eadi 
component  there  will  be  one  mqjor  field  agency  separate  and  distinct  fimm  the 
developin^procuring  command  and  firom  the  using  command  whicdi  will  be 
responsible  for  OT&E...."*® 

The  Air  Staff  OT&E  directorate  (XOOW),  whidi  had  42  personnel  in 
early  1974,  was  reduced  in  scope  to  become  a  7-person  division  (AF^OOFA, 
later  XOORE  and  XOOST)  responsible  mainly  for  coordination.*^  The  Air 
Force’s  msgor  commands,  however,  continued  to  conduct  most  OT&E  programs 
and  many  related  activities,  such  as  TD&E.  As  a  result,  many  of  the 
commands  continued  to  operate  their  own  test  units  as  separate  OTAs  (see 
pages  20-21).  This  posed  a  potential  inconsistency  with  DoD  guidance  that 
would  be  not  be  addressed  for  another  17  years. 


*To  more  clearly  indicate  its  mission  and  avoid  confusion  with  DT&E,  the 
word  "operational"  was  added  to  the  Center's  name  in  April  1983. 
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Evolution  of  AFOTEC.  1974- 1991 


The  Air  Force  Test  and  Evaluation  Center  was  activated  as  a 
sepiuute  operating  agency  (SOA)  on  1  January  1974  at  Kirtland  AFB, 
Kirtland  was  already  site  of  the  Air  Force  T&E  Systems  Program  Office 
(TESPO),  assigned  to  AFSC’s  Air  Force  Special  Weapons  Center,  which  was 
developing  a  vast  Continental  Operating  Range  (COR)  in  the  Great  Basin  area, 
initially  for  the  Tactical  Air  Command.  AFTEC  was  slated  eventually  to  oper¬ 
ate  the  COR,  which  had  evolved  from  the  Have  Edge  concept  of  the  late  1960s, 
but  Congress  did  not  fund  the  project  in  1974,  and  the  TESPO  was  soon 
disbanded.^ 

The  people  assigned  to  AFTEC  in  the  neirt;  several  months  had  to  "hit 
the  ground  running."  Maj  Gen  John  M.  Bums  assumed  command  on  25 
February.  The  Center  achieved  initial  operational  capability  in  April  1974, 
and  became  fully  operational  in  October  1974.  By  yeaFs  end  it  was  managing 
32  OT&E  programs,  leading  two  DoD-sponsored  JT&E  programs,  and 
monitoring  103  operational  tests  being  conducted  by  the  MAJCOMs. 

Organization 

Unlike  the  large  and  cumbersome  Air  Proving  Ground  Command, 
AFTEC  was  designed  as  a  small  management  headquarters  that  would  borrow 
most  of  the  equipment,  personnel,  and  facilities  needed  for  field  testing  fimm 
the  developing  and  operating  commands,  \ifith  termination  of  the  continental 
operating  range  project,  AFOTEC  remained  dependent  on  DT&E  facilities  and 
training  ranges  belonging  to  other  commands  and  services. 

Size.  As  its  workload  of  programs  and  other  responsibilities  grew 
during  the  next  decade,  AFOTEC  expanded  to  over  600  personnel  in  the 
headquarters  and  field.  This  number  remained  fairly  stable  firom  1984  through 
1991.  In  addition  to  the  people  assigned  to  AFOTEC,  the  commander  also 
exercised  operational  control  over  a  comparable  number  attached  to  test  teams 
and  provided  guidance  to  as  many  as  2,000  personnel  involved  in  operational 
testing  by  the  MAJCOMs.^ 

Headquarters.  From  1974  until  1992,  AFOTECfs  headquarters 
organization  followed  a  matrix  structure  in  which  each  OT&E  program  drew 
upon  a  variety  of  personnel  fix)m  functional  areas  in  several  directorates. 
These  specialists  formed  test  support  groups  (TSGs),  headed  by  a  test 
manager,  to  plan,  coordinate,  and  support  individual  OT&E  programs.  Most 
test  managers  were  assigned  to  one  of  several  division  in  the  Directorate  of 
Test  and  Evaluation  (TE).  The  director  of  TE  also  supervised  detachments  and 
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test  teams  in  the  field,  which  normally  did  the  actual  testing.*  Figure  3  on 
page  31  shows  the  final  version  of  this  organizational  structure  in  detail. 

Field  Units.  Since  most  tests  could  not  be  conducted  fi:om  Kirtland, 
AFOTEC  formed  test  teams  as  geographically  separate  operating  locations 
(OLs)  whenever  and  wherever  needed.  At  several  key  locations  (see  map  at 
Figure  2),  AFOTEC  established  detachments  (dets)  to  support  test  teams  and 
administer  field  activities. 

-  Pet  1.  first  located  at  Kapaun  AS,  Germany,  was  activated  in  Jime  1977  to 
provide  liaison  with  United  States  Air  Forces  in  Europe  (USAFE)  at  nearby 
Ramstein  Air  Base  and  other  European  organizations.  As  a  result  of  the  post- 
Cold  War  drawdown  and  fewer  tests  being  conducted  in  Europe,  Det  1  was 
inactivated  and  replaced  by  OL-RC  on  30  September  1991.  Just  nine  months 
later,  Det  1  was  reactivated  at  Scott  AFB,  IL,  as  AFOTEC’s  major  field  \init 
for  command,  control,  communications,  and  computer  testing  (see  page  22). 

>  Det  2.  at  Eglin  AFB,  FL,  was  activated  in  August  1977.  It  worked  closely 
with  local  AFSC  and  Tactical  Air  Command  organizations  at  this  major  T&E 
complex,  such  as  the  Tactical  Air  Warfare  Center  (TAWC)  and  the  former 
Armament  Division.  Det  2  supported  numerous  teams,  most  involved  with 
munitions  and  electronic  combat.  In  1992  its  mission  and  organization 
expanded  as  it  absorbed  much  of  the  operational  testing  workload  of  the  Air 
Combat  Command’s  Air  Warfare  Center  (AWC). 

-  Det  3.  at  Nellis  AFB,  NV,  was  activated  in  April  1978  to  interface  with  the 
Tactical  Fighter  Weapons  Center  (TFWC)  and  other  organizations  at  the  Nellis 
range  complex.  It  later  shrank  in  size,  largely  as  a  result  of  TFWC’s  reduced 
OT&E  mission,  and  was  replaced  by  OL-TF  on  30  September  1991. 

-  Det  4.  whidi  had  been  the  designation  of  a  classified  unit  working  on  mobile 
MX  missile  basing  fimm  August  1980  through  December  1981  at  IQrtland,  was 
reactivated  in  Colorado  Springs  in  February  1984  to  support  space-related  test 
teams  and  serve  as  AFOTEC’s  liaison  with  US  and  Air  Force  Space  Commands 
as  well  as  the  North  American  Air  Defense  Command  (NORAD).  Det  4  later 
moved  on  to  Peterson  AFB  and,  in  1992,  took  over  much  of  the  Space 
Command  OT&E  mission  it  had  been  monitoring. 


*For  a  stunmaiy  of  organization  changes  in  the  headquarters  culminating 
with  the  complete  reorganization  in  1992,  see  pages  32-34. 
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•  Pet  5.  at  Edwards  AFB,  CA,  was  activated  in  July  1982.  This  was  eight 
years  after  AFTEC  first  proposed  a  detachment  there,  an  idea  that  was  not 
supported  by  Systems  Command  at  the  time.  Pet  5’s  main  mission  was  to 
serve  as  the  parent  organization  for  the  numerous  aircraft  and  missile  test 
teams  that  had  been  working  at  the  Air  Force  Flight  Test  Center  (AFFTC) 
since  AFOTEC's  earliest  days.^ 


Figure  2.  Location  of  AFOIEC  Detachmenta 


OTftE  Procedurea 

Althoui^  the  detailed  methods  used  for  OT&E  have  continually 
evolved  throughout  AFOTECs  histoiy,  the  following  basic  activities  and  their 
undeilying  policies  and  procedures  have  remained  fairly  consistent^ 

Planning.  A  common  theme  during  AFOTECs  history  has  been 
ever«ailier  participation  in  acquisition  programs  and  definition  of  the  user 
requirements  that  normally  generate  new  qrstems.  After  program  definition, 
AFOTEC  begins  advance  planning.  This  leads  to  development  of  a  test 
concept,  whidi  throuid^  a  test  planning  review  (TPR)  process,  evolves  during 
pre-test  planning  into  a  detailed  OT&E  plan.  Throughout  thia  process 
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AFOTEC  also  prepares  the  OT&E  section  of  successive  test  and  evaluation 
master  plans  (TEMPs),  which  also  address  DT&E.  Based  on  basic  program 
documentation,*  AFOTEC’s  test  plan  identifies  all  the  various  elements  to  be 
evaluated,  including  critical  operational  issues  (COIs),  measures  of  effective¬ 
ness  (MOEs),  test  objectives,  measiu'es  of  performance  (MOPs),  and  evaluation 
criteria.  The  plan  also  covers  all  aspects  of  the  coming  OT&E  effort,  including 
resources  and  support  needed  (which  other  documents  break  out  in  detail),  test 
scenarios,  schedule,  planning  considerations,  and  limiting  factors. 

Limitations.  Resource  consfi’aints  (e.g.,  the  infeasibility  of 
replicating  actual  warfare)  has  limited  the  scope  of  almost  all  tests.  Beyond 
these  inherent  limitations,  AFOTEC  has  attempted  to  test  as  realistically  as 
practical  but  historically  faced  other  limiting  factors.  Some  of  the  most  com¬ 
mon  were  inadequate  threat  systems,  safety  and  airspace  restrictions,  less 
than  optimal  instrumentation,  too  few  primary  test  artides,  not  enough 
support  equipment,  the  system  tested  not  being  production-representative,  use 
of  immature  software,  insuffident  maintenance  data,  and  incomplete 
documentation.^^ 

The  insuffidency  of  existing  ranges  and  test  facilities  has  been  a 
persistent  problem.  In  1984,  several  years  after  cancellation  of  the  Continental 
Operating  Range  (see  page  11),  AFOTEC  took  the  lead  in  trying  to  meet  DoD’s 
range  and  threat  limitations  by  initiating  a  statement  of  operational  need  to 
establish  an  Electronic  Combat  Test  Capability  (ECTC)  program.  If  the 
program  had  continued,  ECTC  would  have  upgraded  the  Utah  Test  and 
Training  Range  to  provide  an  integrated  air  defense  system.  Cuts  in  the 
defense  budget  and  other  factors  led  to  the  cancellation  of  the  ECTC  program 
in  1990,  but  AFOTEC  has  continued  to  work  actively  for  improved  test 
capabilities  and  resources.^ 

Test  Execution.  At  least  nine  months  before  a  scheduled  start  date, 
the  test  director  is  normally  assigned  to  the  detachment  or  OL  at  the  main  test 
location  and  begins  to  build  the  test  team  to  conduct  the  OT&E.  In  many 
lOT&Es,  AFOTEC’s  teams  have  shared  resources  with  a  previously  established 
DT&E  team  as  part  of  a  combined  test  force  (CTF)."  Led  by  the  Responsible 
Test  Organization  (RTO),  the  CTF  could  also  include  representatives  of  the 


*  These  documents  and  their  names  dianged  over  the  years.  For  example, 
the  statement  of  operational  need  (SON)  and  the  systems  operational  require¬ 
ments  document  (SORD)  became  the  operational  requirements  document 
(ORD)  in  early  1991.  Other  key  documents  have  included  the  program 
management  directive  (PMD),  the  system  threat  assessment  report  (STAR)  and 
the  recently  emphasized  cost  and  operational  effectiveness  analysis  (COEA). 
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System  Program  Office  (SPO)  and  contractors  developing  the  system.  In  cases 
of  multiservice  OT&E,  the  participating  services’  OTAs  establish  a  joint  test 
organization  under  the  designated  lead  service’s  test  director.  In  any  case,  the 
OT&E  team  processes  and  interprets  test  data  with  complete  independence 
from  its  DT&E  countexparts. 

In  almost  all  cases,  the  MAJCOMs  have  loaned  AFOTEC  the  bulk  of 
the  personnel  and  equipment  needed  to  conduct  its  tests.  To  help  determine 
operational  suitabilily  through  use  of  a  common  data  system,  the  CTF  forms 
a  Joint  Reliability  and  Maintainability  Evaluation  Team  ( JRMET).  Test  teams 
also  submit  product  quality  defidency  reports  (PQDRs-known  also  as  "service 
reports"  prior  to  1991)  to  identify  defidendes  discovered  in  testing  and 
recommend  enhancements,  with  the  higher  priority  PQDRs  implemented  as 
soon  as  possible.""  Many  of  the  personnel  who  take  part  in  the  OT&E  later 
become  available  to  help  the  MAJCOM  in  the  "beddown"  and  initial  operation* 
al  capability  (IOC)  of  the  system. 

Axudj^is.  The  main  focus  of  most  test  activities  has  been  the 
generation  of  data  (e.g.,  statistics,  telemetry,  visual  documentation,  and 
questionnaires).  To  find  trends  and  draw  valid  condusions  requires  data 
collection  firom  as  many  sources  or  perspectives  as  practical,  reduction  of  this 
data  to  a  useable  volume  while  reviewing  for  consistency  and  discrepandes 
(e.g.,  "outlying  data"),  and  examination  of  the  reduced  data  for  spedfic 
objectives.  For  tests  involving  large  volumes  of  data,  team  analysts  and 
Headquarters  AFOTEC-augmented  by  support  contractors-use  automated 
data  reduction  and  analysis  packages  to  hdp  "crunch"  the  numbers. 


Effectiveness 

Security 

Vulnerability 

Susceptibility 

Survivability 

Interoperability 

Capability 

Table  1.  Some  Attributes 


Suitability 

Reliability 

Transportshility 

Compatibility 

Maintainability 

Supportability 

Availability 

Iveness  and  Suitability 


As  shown  in  the  table  above,  these  analyses  address  the  main 
characteristics  of  the  tystem,  with  operations  analysts  trying  to  answer  the 
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question  "is  it  capable?"  and  logistics  analysts,  the  question  "will  it  be  avail¬ 
able?"  In  the  increasingly  important  field  of  software  suitability,  AFOTEC  has 
also  developed  considerable  expertise. 

Modeling  and  Simulation.  Supported  by  an  expanding  Commun- 
icationS’Computer  Systems  Division  (SC),  AFOTEC’s  analysts  built  up  consid¬ 
erable  experience  in  modeling  and  simulation  (both  digital  and  hybrid  or  "man- 
in-the-loop")  during  the  1980s.  AFOTEC  developed  its  own  computer  models 
as  well  as  using  other  DoD  facilities  and  the  expertise  available  through 
technical  support  contracts.  The  use  of  modeling  and  simulation  could  be 
especially  valuable  during  EOAs  and  OUEs.  Although  noted  for  its  expertise 
in  this  area,  the  Center  consistently  followed  a  firm  policy  that  modeling  and 
simulation  can  supplement  but  not  substitute  for  actual  testing.'^ 

Reporting.  AFOTEC’s  most  important  product  has  not  necessarily 
been  testing  or  analysis  per  se,  but  rather  the  information  contained  in 
briefings  presented  and  reports  published  on  the  results.  In  addition  to  end-of- 
test  briefings  and  detailed  final  reports,  lengthy  OT&E  programs  have  often 
required  interim  summary  reports  as  well  as  periodic  briefings  and  other 
assessments.  The  briefings  and  reports  go  to  a  wide  range  of  officials  and 
organizations,  such  as  SPOs,  Program  Executive  Officers  (PEOs),  MAJCOMs, 
the  Air  Staff,  other  OTAs,  and  DOTd£.  The  briefings  or  portions  thereof  are 
also  often  presented  to  the  AFSARC  or  DAB. 

In  its  first  few  years,  AFOTEC’s  reports  were  fiurly  unstructured  in 
their  approach  and  format.  Starting  in  1977,  they  became  more  compact  and 
consistent,  although  continuing  to  present  results  in  a  variety  of  ways  and 
often  directly  recommending  whether  to  produce  or  not  produce  the  system 
tested.  After  this  practice  ended  in  1983,  OT&E  reports  were  assigned 
Inspector  General-style  rating  terms,  such  as  "outstanding,"  "satisfactory," 
"marginal,  and  "unsatisfactoiy."  AFOTEC  dropped  use  of  these  judgmental 
words  in  early  1988  and  began  stating  its  evaluation  results  strictly  on 
whether  systems  met  or  did  not  meet  user  requirements  or  criteria  in  specific 
areas.  In  1992,  consistent  with  DOT&E’s  philosophy  and  the  practices  of  the 
other  service  OTAs,  AFOTEC  began  to  use  higher-level  measures  and  draw 
overall  conclusions  about  the  system’s  effectiveness  and  suitability.  Decision¬ 
makers,  however,  still  must  wei^  the  OT&E  results  with  other  factors,  such 
as  the  system’s  potential  for  improvement,  past  contractor  performance, 
budgetary  constraints,  cmrent  force  structure  needs,  the  latest  threat  projec¬ 
tions,  alternative  solutions,  and  economic  or  political  realities.*’ 

After  termination  of  an  OT&E  program,  the  final  report  and  other 
key  documents  are  preserved  as  a  permai^nt  historical  record  on  the  proce¬ 
dures  and  findings  of  the  OT&E.  Starting  in  the  early  198()s,  AFOTEC  also 
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began  to  maintain  an  OT&E  Lessons  Learned  database  of  suggestions  sub¬ 
mitted  by  test  teams  and  personnel  at  the  headquarters.^ 

AFOTEC  Commanders  and  Policies 

Although  the  evolution  of  OT^  polipy  and  procedures  has  been 
strongly  influenced  by  outside  factors,  past  commanders  of  AFOTEC  have  left 
their  mark  as  well.  They  have  also  helped  shape  the  Center’s  philosophy, 
mission,  and  mode  of  operation-as  indicated  in  the  brief  chronology  that 
follows.** 

Early  Commanders,  1974-76.  V^th  four  commanders  in  the  first 
three  years  (see  Appendix  B),  AFTEC  devoted  most  of  its  attention  to  starting 
and  conducting  tests-many  of  which  had  first  been  planned  before  its 
formation-and  asserting  its  basic  authorily  and  independence  in  the  face  of 
reluctant  acquisition  and  T&E  communities.**  Revisions  of  AFR  23-36 
(AFTEC*s  mission  regulation)  and  AFR  80-14  ("Test  and  Evaluation")  in  July 
1976  helped  give  the  new  Center  more  dout  while  work  began  on  a  new  Air 
Force  OT&E  directive. 

Maj  Gen  Howard  W.  Lea^  1976-80.  General  Leaf,  whose  T&E 
experience  had  begun  as  a  test  pilot  at  Eglin  in  1955,  commanded  AFTEC 
during  this  crucial  period  when  the  Center  almost  Rubied  in  size  while 
continuing  to  assert  its  independence  and  build  its  oredibilily.**  After  a 
lengthy  coordination  process,  the  Air  Force  published  AFM  55-43  on  OT&E 
management  in  June  1979.  Althou^  not  entirely  mandatory,  this  AFTEC- 
originated  manual  provided  detailed  guidance  to  the  MAJCOMs.  A  revised 
AFR  57-1  on  operational  requirements,  also  published  in  June  1979,  formalized 
AFTEC’s  review  of  documentation  on  proposed  new  systems.  AFTEC  initiated 
a  deficiency  reporting  process  and  formation  of  the  Air  Force  OT&E  Resources 
Management  Assessment  System  (the  ORMAS)-a  corporate  body  for 
programming  funds,  ranges,  personnel,  test  assets,  and  other  support  among 
the  Air  Staff  and  MAJCOMs.  Internally,  the  Center  established  a  senior 
advisory  committee  and  began  more  systematic  advance  planning,  threat  real¬ 
ism,  suitability  analysis,  software  evaluation,  and  test  reporting.  It  also 
conducted  the  first  mqjor  OT&E  based  on  a  concept  of  models,  simulation,  and 
flight  testing.  In  ^ect,  AFTEC  assumed  the  basic  structure  and  established 
the  procedures  followed  for  the  next  dozen  years. 

M^f  Gen  Wayne  E.  Whidatch,  1980-82.  By  the  time  General 
Whitlatch  assumed  command,  AFTEC  had  reached  the  stage  of  maturity  when 
it  was  appropriate  to  step  bac^  and  do  some  in-depth  self-examination.  One 
year  after  his  arrival  General  Whitlatch  used  the  results  of  two  xnqjor  stuches 
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of  AFTEC’s  mission  and  organization  to  write  a  comprehensive  mission  state¬ 
ment  compatible  with  the  latest  strategic  planning  concepts. 

M^j  Gen  Richard  W.  Phillips,  1982-85.  Under  the  command  of 
General  Phillips,  AFOTEC  replaced  AFM  55-43  with  a  shorter  APR  55-43, 
published  in  June  1985.  Although  directive  in  nature,  the  regulation  still 
allowed  considerable  leeway  to  the  MAJCOMs  in  how  to  plan,  conduct,  and 
report  OT&E.  General  Phillips  implemented  the  IG-style  rating  system  and 
an  early  type  of  system  examination  known  as  lOT&E  Phase  1  to  help  meet 
the  information  needs  of  decision-makers  prior  to  Milestone  III  of  the 
acquisition  (^de. 

M^j  Gen  Michael  D.  Hall,  1985-87.  During  the  tour  of  General 
Hall,  who  had  once  headed  its  AIM-9  Sidewinder  test  team,  the  Center 
updated  the  official  mission  statement  in  AFR  23-36,  published  in  March  1986. 
The  revised  regulation  spelled  out  several  functions,  such  as  the  various  ele¬ 
ments  that  comprise  operational  smtability,  not  covered  in  the  previous  1980 
edition.  In  the  continuing  effort  to  find  a  way  to  provide  earlier  inputs  to 
dedsion-makers,  he  redefined  IOT&E(l)  as  the  early  operational  assessment 
(EGA).  As  an  overall  theme  during  his  command.  General  Hall  emphasized 
making  OT&E  an  "integral  part  of  the  acquisition  process"  and  improving 
relations  with  developers.  He  also  began  holding  offsite  meetings  to  encourage 
fi^sh  looks  at  the  way  the  Center  did  business-one  result  of  which  was  an  in- 
depth  organizational  assessment  by  a  team  of  outside  consultants,  published 
in  September  1987. 

Mqj  Gen  Cecil  W.  Powell,  1987-89.  General  Powell,  who  came  to 
AFOTEC  in  July  1987  with  a  broad  range  of  T&E  experience  and  strong 
backing  finm  the  CSAF,  changed  the  Air  Force's  evaluation  policy  finm 
assigning  IG-style  ratings  to  predse  descriptions  of  test  results  based  on 
meeting  or  not  meeting  specific  user  requirements.  He  led  an  effort  to 
standardize  OT&E  procedures  Air  Force-wide-a  goal  largely  met  with  a 
thorough  revision  of  AFR  55-43  first  issued  in  September  1989  and  formally 
published  in  Jime  1990.  General  Powell  also  stressed  early  and  disdplined 
planning  (resulting  in  the  Test  Planning  Review  or  TPR  process),  initiated  the 
DoD-wide  definition  for  operational  assessments  (OAs  and  EGAs),  and  helped 
reinforce  AFOTEC's  longstanding  philosophy  that  modeling  and  simiilation 
should  supplement  rather  than  substitute  for  test  data.  He  took  a  spedal 
interest  in  improving  the  Air  Force’s  requirements  process  and  was  a  key 
contributor  to  the  Air  Force’s  first  acquisition  program  corporate  review  (a 
B-IB  "summit  meeting")  in  recent  history. 

Mi^  Gen  Peter  D.  Robinson,  1990-91.  General  Robinson’s  solid 
analytical  background  brou^t  increased  attention  to  the  complex  processes 
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underlying  the  measurements  of  effectiveness  and  suitability  and  the  need  to 
reduce  "undetermined"  ratings.  He  also  institutionalized  the  concept  of  stra¬ 
tegic  planning  and  introduced  the  philosophy  of  total  quality  (TQ)  to  the  way 
AFOTEC  performed  its  staff  work  and  served  its  suppliers  and  customers~who 
include  both  the  users  of  the  equipment  tested  and  the  decision-makers  relying 
on  the  Center^s  reports.  AFOTECs  independence  was  confirmed  when,  on  5 
Februaxy  1991,  it  became  a  direct  reporting  unit  (DRU)  while  most  other  SO  As 
became  field  operating  agencies  (FOAs). 

Mi^  Gen  Marcus  A.  Anderson,  1991%  As  described  in  the  final 
sections  of  this  histoiy  (pages  28-34),  Mqj  Gen  Marcus  A.  Anderson  led 
AFOTEC  through  the  most  significant  period  of  change  since  the  Center’s 
establishment.  He  oversaw  the  implementation  of  TQ  and,  at  a  time  when  the 
entire  Air  Force  was  realigning  and  downsizing,  he  completely  revamped  the 
Center’s  organization  and  expanded  its  mission  to  encompass  most  of  the 
OT&E  traditionally  performed  by  the  mqjor  commands.  Under  his  leadership, 
AFOTEC  also  raised  the  focus  of  its  evaluations  to  mission  level  measures  and 
hi^er  level  ratings  and,  in  the  face  of  post-Cold  War  budget  cuts,  began 
looking  at  ways  to  make  OT&E  as  cost-effective  as  possible. 


OT&E  and  the  Major  Commands 

Prior  to  the  dianges  of  1992,  the  operational  Mqjor  Commands- 
espedally  the  Tactical  Air  Command  (TAC),  Strategic  Air  Command  (SAC), 
Military  Airlift  Command  (MAC)-had  long  conducted  a  wide  variety  of  OT&E 
programs  on  new  and  modified  systems  in  their  inventories.  Except  for 
systems  unique  to  their  theaters,  USAFE  and  Pacific  Air  Forces  (PACAF) 
normally  relied  on  TAC  to  perform  operational  testing  for  all  the  tactical  air 
forces  (TAF).  This  extensive  OT&E  program  came  under  the  purview  of  TAC’s 
DCS  for  Requirements  (TAC!/DR),  while  that  of  SAC  and  MAC  were  managed 
by  their  DCSs  for  Plans  and  Requirements  (XR). 

The  other  commands  also  managed  OT&E  programs.  Air  'Training 
Command  (ATC)  focused  on  training  aircraft  and  related  systems.  Air  Force 
Space  Command  (AFSPACECOM)  began  conducting  OT&E  of  space  systems 
in  the  mid-1980s,  and  the  Air  Force  Special  Operations  Command  (AFSOC) 
took  over  testing  of  its  systems  firom  MAC  during  the  first  part  of  1991.  The 
Electronic  Security  Command  (ESC)  at  Kdly  AFB,  TX,  whidi  expanded  to 
become  the  Air  Force  Intelligence  Command  (AFIC)  on  1  October  1991,  tested 
cryptographic  systems  and  sensitive  intdligence  equipment.  Air  Force 
Communications  Command  (AFCC),  which  had  conducted  OT&E  of  communi- 
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cations,  air  traffic  control,  and  computer  equipment,  transferred  many  of  its 
field  units  to  the  MAJCOMs  in  October  1990  and  was  reduced  to  an  FOA  in 
July  1991.  Besides  T&E  offices  at  their  headquarters,  several  of  the 
MAJCOMs  had  field  units  that  performed  OT&E. 

MAJCOM  Field  Units 

TAG.  Because  the  Tactical  Air  Command  was  responsible  for  the 
requirements,  training,  and  employment  of  the  vast  majority  of  actual 
warfighting  systems,  it  had  the  largest  OT&E  program  of  any  of  the  major 
commands.  It  established  the  USAF  Tactical  Air  Warfare  Center  (TAWC)  in 
1963  to  carry  on  testing  of  Air  Force  versus  Army  concepts  for  dose  air  support 
and  tactical  airlift.  With  expansion  of  the  Vietnam  War,  TAWUs  charter  grew 
to  indude  a  wide  range  of  development,  testing,  and  tactics  activities-with  a 
spedal  emphasis  on  electronic  combat.  In  1966  TAC  created  the  Tactical 
Fighter  Weapons  Center  (TFWG)  at  Nellis  AFB,  NV,  and  the  Tactical  Airlift 
Center  at  Pope  AFB,  NG,  to  share  the  increasing  workload.  It  also  assigned 
OT&E  to  a  Tactical  Air  Reconnaissance  Center  and  a  Spedal  Operations  Force. 
In  July  1971  TAC  consolidated  most  OT&E  at  TAWC.  MAC  inherited  the 
Airlift  Center  when  it  assumed  the  tactical  airlift  mission  firom  TAC  in  1974. 
Both  TAWC  and  TFWC  (which  focused  on  TD&E),  plus  their  subordinate 
wings,  groups,  and  squadrons  also  became  increasinidy  involved  in  extensive 
training  programs,  such  as  Blue  Flag  and  Red  Flag  exerdses.^^  On  1  October 
1991  the  word  "tactical”  was  dropped  firom  the  names  of  the  two  centers,  with 
TAWC  becoming  the  Air  Warfare  Center  and  TFWC  later  becoming  the 
Weapons  and  Tactics  Center. 

SAC.  Within  the  Strategic  Air  Command,  OT&E  activities  were 
divided  among  several  squadrons.  The  31st  Test  and  Evaluation  Squadron  at 
Edwards  AFB  was  activated  on  1  July  1986  to  assume  the  testing  mission  of 
the  4200d  TES  which  had  begun  operations  in  the  mid-1960s.  It  focused  on 
testing  SAC  aircraft  and  related  systems.  The  49th  Test  Squadron  (TESTS) 
at  Barksdale  AFB,  LA,  began  in  July  1972  as  the  4201st  TESTB  at  Pease  AFB, 
NH.  It  moved  to  Barksdale  in  1974  and  was  designated  as  the  49th  TESTS 
in  July  1986.  The  squadron  focused  on  air-launched  missile  and  munitions 
testing  and  certification.  The  513  TESTS  was  activated  in  July  1986  at  Ofiutt 
AFB,  NE,  where  it  concentrated  on  SAC*s  electronic  warfare  systems  and 
software.  The  576  TEST  at  Vandenberg  AFB,  CA,  tested  SAC*s  ballistic 
missiles.  The  31st,  49th,  and  513th  squadrons  reported  directly  to  SAC^s  DCS 
for  Requirements  and  Test  (XR),  while  the  576th  was  part  of  the  1st  Strategic 
Aerospace  Division,  whidi  became  20th  Air  Force  less  than  a  year  before  SAC 
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was  disestablished  in  June  1992.”  Most  of  SAC’s  test  units,  including  the 
99th  Test  Group  at  OfiEiitt  AFB,  NE,  were  integrated  into  the  Air  Combat 
Command  and  its  two  centers. 

MAC.  The  USAF  Airlift  Center  (ALCENT)  at  Pope  AFB,  NC,  was 
transferred  from  TAC  to  MAC  in  1974  with  the  transfer  of  the  tactical  airlift 
mission.  It  focused  on  the  testing  of  equipment  and  systems  related  to  air 
transport  and  worked  closely  with  Army  units  at  nearby  Fort  Bragg.  The 
ALCENT  was  renamed  the  Air  Mobility  Center  to  reflect  the  formation  of  the 
Air  Mobility  Command  in  June  1992. 

AFSOC.  The  Military  Airlift  Command  created  the  Special  Missions 
Operational  Test  and  Evaluation  Center  (SMOTEC)  at  Hurlburt  Field,  FL,  in 
October  1983.  It  was  formed  by  consolidating  the  combat  rescue  test  hmctions 
formerly  performed  by  the  1550th  Aircrew  Training  and  Test  Wing  at  Kiitland, 
and  the  special  operations  test  functions  of  the  2nd  Air  Division  and  its  1st 
Special  Operations  Wing  at  Hurlburt  SMOTEC  became  a  unit  of  the  Air 
Force  Special  Operations  Command  when  AFSOC  was  established  in  May 
1990.  In  addition  to  testing  systems  used  in  special  operations  and  low- 
intensity  conflict,  the  Center  was  also  involved  in  requirements  and  doc¬ 
trine.” 

AFCC.  The  Air  Force  Commimications  Service  activated  the  1815th 
Test  Squadron  at  Richards  Gebaur  AFB,  MO,  in  January  1973  to  do  operation¬ 
al  testing  of  new  communications  and  air  traffic  control  equipment.  In  Jime 
1981  the  1815th  was  moved  to  Wright-Patterson  AFB,  OH,  and  combined  with 
personnel  from  the  Creek  Scope  program,  which  was  updating  long-haul 
communications  in  Europe.  The  1815th,  which  was  renamed  an  OT&E 
Squadron  in  November  1985,  became  the  Air  Force  Communications  Command 
OT&E  Center  (AFCC  OTEC)  in  April  1989.  As  the  result  of  a  Defense 
Management  Report  initiative,  the  OTEC  was  reorganized  as  the  T&E  Division 
of  AFCC’s  new  Technology  Integration  Center  (TIC!/TE)  at  Scott  AFB,  IL,  in 
April  1991.”  Here,  on  1  June  1992,  it  became  AFOTEC’s  new  Det  1  (see  page 
13). 

Monitored  Tests 

Headquarters  AFOTEC  and  its  detachments  monitored  and  advised 
on  the  OT&E  programs  conducted  by  MAJCOMs  and  their  subordinate 
organizations.  Monitored  OT&Es  were  defined  as  USAF-directed  tests 
involving  high  risk,  hi^  cost,  and  priority  precedence  systems.  For  internal 
management,  AFOTEC  designated  them  as  Categozy  1  tests.  On  these 
AFOTEC  coordinated  on  the  TEMP,  approved  test  plans,  and  commented  on 
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final  reports.  For  smaller  and  less  critical  programs,  AFOTEC’s  role  was 
advisory.  These  OT&Es  included  USAF-directed  (Category  2)  or  command- 
initiated  (Category  3)  projects.  At  the  end  of  1991,  AFOTEC’s  Test  Resources 
and  Information  Management  System  (TRIMS)  listed  106  Category  1,  227 
Category  2,  and  209  Category  3  tests  being  conducted  by  the  MAJCOMs-a 
total  of  542  (compared  to  585  one  year  earlier).  AFOTEC  planners  also  used 
Categories  4,  5,  and  6  for  tracking  conducted,  monitored,  and  cognizance 
programs  projected  for  the  fiiture.^^ 

Developers  and  Implementors 

Air  Force  Systems,  Communications,  and  Logistics  Commands 
(AFSC,  AFCC,  and  AFLC)  had  long  been  responaibie  for  developing,  acquiring, 
implementing,  and  supporting  the  new  and  modified  systems  tested  by 
AFOTEC  and  the  MAJCOMs.  During  AFOTEC’s  first  decade,  it  ofien  found 
itself  in  an  adversarial  relationship  with  AFSC  and  its  SPOs.  System 
Command’s  renunciation  of  its  traditional  role  as  the  advocate  of  new 
programs  in  1989  and  its  loss  of  mcgor  system  responsibilities  to  the  newly 
created  program  executive  officers  (FEOs)  in  early  1990  changed  the  context 
of  AFOTEC’s  relationship  with  developers.  The  replacement  of  AFSC,  AFLC, 
and  much  of  AFCC  by  the  new  Air  Force  Materiel  Command  (AFMC)  on  1  July 
1992  then  consolidated  the  command  structure  with  which  AFOTEC  works. 

Informal  Contacts 

Many  of  AFOTEC’s  contributions  to  the  other  commands’  OT&E 
programs  as  well  as  testing-related  aspects  of  acquisition  programs  took  place 
infoimally  at  the  action  officer  level.  The  directors,  chief  scientist,  vice 
commander,  and  commander  also  tried  to  resolve  the  more  difficult  issues 
through  fi'equent  personal  meetings  and  phone  conversations  with  other 
officials.  Sometimes  AFOTEC  served  as  an  intermediary  between  developers 
and  users  on  various  issues  related  to  requirements  and  testing.  Although 
usually  undocumented,  the  Center’s  informal  contacts  represented  an  extensive 
and  integral  part  of  AFOTEC’s  management  responsibilities.'^ 
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OT&E  Outside  the  Air  Force 


Other  Services 

As  with  many  other  functions,  each  of  the  armed  forces  had  devel¬ 
oped  its  own  approach  to  performing  OT&E.  They  all,  however,  have  operated 
under  the  principle  that  the  uniformed  services  should  be  responsible  for 
operationally  testing  the  weapons  that  they  will  have  to  fight  with  in 
combat.*® 

Unlike  the  Air  Force,  which  did  away  with  its  Air  Proving  Ground 
Command  in  1957,  the  Navy  has  kept  an  operational  testing  organization 
intact  since  World  War  n.**  This  organization  became  the  DoD’s  first 
independent  operational  test  agency  in  1971,  one  year  before  the  Army  estab¬ 
lished  its  coimterpart.  The  Marine  Corps  activated  its  OTA  in  1978.**  To 
coordinate  policies  and  discuss  issues  of  common  concern,  the  four  OTAs 
periodically  hold  commanders*  conferences. 

Army.  On  8  November  1990,  the  Army  activated  a  new  Operational 
Test  and  Evaluation  Command  (OPTEC)  to  replace  and  expand  upon  its 
Operational  Test  and  Evaluation  Agency  (OTEA),  which  had  served  as  the 
Army’s  OTA  since  being  established  at  Fort  Belvoir,  VA,  in  September  1972.** 
Formation  of  OPTEC  implemented  a  Defense  Management  Report  decision  of 
20  November  1989  to  consolidate  Army  OT&E  activities  under  a  single 
command.  The  realignment  allowed  the  Army  to  reduce  the  number  of  person¬ 
nel  involved  with  OT&E  firom  approximately  2,700  to  2,000  positions.*^ 

In  addition  to  its  headquarters  in  Alexandria,  VA,  the  new  OPTEC 
encompassed  several  components:  the  Operational  Evaluation  Center  (OEC), 
which  included  former  OTEA  functions  at  Alexandria;  the  OPTEC  Threat  Sup¬ 
port  Activity  (OTSA),  formerly  the  Army  Development  and  Acquisition  Threat 
Simulators  Activity  (ADATS-A),  at  Fort  Bliss,  TX;  and  the  Test  and  Experi¬ 
mentation  Command  (TEXCOM),  formerly  a  subcommand  of  the  Training  arid 
Doctrine  Command  (TRADOC),  headquartered  at  Fort  Hood,  TX.  TEXCOM 
continued  to  operate  its  Experimental  Center  (the  TEC)  at  Fort  Hunter  Ligget, 
CA,  but  OPTEC  replaced  TRADOC’s  traditional  test  boards  with  directorates 
and  T&E  Coordinating  Offices  (TECOs)  responsible  for  various  combined  arms 
centers  and  branches:  Infantry,  Signal,  Armor/Engineer,  Aviation,  Fire 
Support,  Intelligence,  and  Communications/Electronics,  Air  Defense  Artillery, 
and  Aiihome/Spedal  Forces.  Among  the  key  positions  in  Army  OT&E  are  the 
evaluators  at  OEC  and  the  test  officers  and  test  directors  at  TEXCOM.  For 
mqjor  tests,  a  senior  officer  firom  the  participating  field  command  serves  as  the 
test  director,  with  a  TEXCOM  officer  as  the  deputy. 
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Prior  to  the  formation  of  OPTEC,  the  Army  relied  on  a  dual  planning 
and  reporting  system  in  which  OTEA  published  independent  test  design  plans, 
independent  evaluation  plans,  and  independent  evaluation  reports,  while 
TRADOC  was  responsible  for  the  test  plans  and  test  reports.  In  line  with  the 
reorganization,  OPTEC  consolidated  documentation  of  both  tests  and  evalua¬ 
tions.  In  addition  to  formal  OT&E,  the  Army  imder  both  OTEA  and  OPTEC 
has  managed  an  open-ended  strategy  known  as  continuotis  comprehensive 
evaluation  (C^).“ 

Navy.  Until  the  formation  of  OPTEC,  the  US  Navy’s  Operational 
Test  and  Evaluation  Force  (OPTEVFOR)-which  traces  its  lineage  back  to 
1945— was  the  largest  of  the  OTAs  in  terms  of  assigned  persoimel  and  test 
resources.  The  commander  (COMOPTEVFOR)  historically  managed  all  OT&E 
programs,  both  large  and  small,  prepared  related  tactics  guides  (a  MAJCOM 
responsibility  in  the  Air  Force),  and  performed  numerous  other  Chief  of  Naval 
Operations  (CNO)  projects.*®  To  perform  these  functions,  OPTEVFOR  (finm 
the  early  1970s  through  the  early  1990s)  had  about  300  people  in  the  Norfolk 
headquarters  and  a  deputate  in  San  Diego  and  over  1,000  personnel  in  the 
field,  including  three  air  test  and  evaluation  squadrons  (AIRTEVRONs):  VX-1 
with  anti-submarine  aircraft  at  Patuxent  River,  MD;  VX-4  with  fighters  at 
Point  Mugu,  CA;  and  VX-5  with  attack  aircraft  at  China  Lake,  CA.  OPTEV¬ 
FOR  has  used  operational  test  directors  (OTDs)  to  perform  the  basic  roles  of 
both  AFOTEC’s  test  managers  and  test  directors.  For  executing  shipboard 
tests,  the  OTDs  normally  deploy  to  the  fleet  and  use  operational  personnel.^ 

The  Navy’s  lOT&E  process  traditionally  has  been  divided  into  OT-I, 
conducted  prior  to  Milestone  II  if  test  articles  were  available,  and  OT-II, 
conducted  prior  to  Milestone  in  and  culminated  by  the  operational  evaluation 
(OPEVAL).  OPTEVFOR  conducts  the  OPEVAL  separate  fi^)m  and  normally 
after  the  technical  evaluation  (TECHEVAL),  which  culminates  DT&E.  OT-III 
is  FOT&E  conducted  after  the  OPEVAL,  and  OT-IV  is  FOT&E  conducted  later 
on  production  systems  if  necessary.^^  In  addition  to  presenting  results  and 
overall  conclusions,  OPEVALs  have  normally  addressed  how  the  test  resolved 
each  of  the  COIs,  if  the  results  support  limited  production,  and  COMOPTEV- 
FOR’s  recommendation  as  to  whether  or  not  the  system  tested  is  ready  for  full 
or  partial  fleet  introduction.^® 

Marines.  The  Marine  Corps  Operational  Test  and  Evaluation 
Activity  (MCOTEA),  a  tenant  organization  with  about  40  personnel  at  Quan- 
tico,  VA,  has  conducted  operational  tests  of  Marine  Corps  systems  and  equip¬ 
ment  since  1978.  Fleet  Marine  Forces  support  MCOTEA  in  executing  tests 
and  providing  the  data  used  by  MCOTEA  to  prepare  independent  evaluation 
reports  (lERs)  for  the  Commandant  of  the  Marine  Corps.  Both  MCOTEA  and 
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OPTEVFOR  also  use  Marine  Helicopter  Squadron  One  (HMX-1)  at  Quantico 
for  tests  involving  rotary  wing  aircraft.  (HMX-1,  formed  in  1947,  is  also  the 
unit  that  flies  government  leaders  in  the  Washington  DC  area.)  Key  Marine 
OT&E  positions  include  operational  test  project  officers  (OTPOs)  at  MCOTEA 
(similar  to  AFOTEC’s  test  managers)  and  test  directors  assigned  to  Fleet 
Marine  Forces,  who  execute  and  report  on  the  tests,  like  AFOTEC  in  the 
past,  MCOTEA  also  monitors  smaller  OT&E  programs  performed  by 
operational  organizations.” 

DoD  Oversight 

Mainly  because  of  dissatisfaction  with  operational  testing’s  influence 
(or  lack  thereof)  on  weapons  acquisition.  Congress  mandated  the  creation  of 
the  Director  of  Operational  Test  and  Evaluation  by  Public  Law  98-94  passed 
in  September  1983.  After  being  established  as  an  OSD  function  in  1984”  and 
having  a  permanent  director  appointed  in  1985,  DOT&E  began  to  play  an 
influential  role  in  the  formulation  of  OT&E  policy.  (The  abbreviation  DOT&E 
refers  to  the  position  but  is  also  used  for  the  organization.)  The  office  grew  to 
about  40  personnel  organized  into  divisions  for  conventional  systems,  strategic, 
C^,  and  resources  and  administration.  Because  of  its  small  size,  DOT&E  came 
to  rely  on  the  Institute  for  Defense  Analyses  (IDA)  to  perform  much  of  its 
technical  work.  Under  its  first  appointed  director,  Mr  John  E.  (Jack)  Krings, 
DOT&E  received  considerable  attention  firom  meodbers  of  the  Congressional 
Militaiy  Reform  Caucus— especially  for  attempts  to  ^qpand  its  role  in  managing 
T&E  resources,  which  they  considered  beyond  the  office’s  statuatory  responsi¬ 
bilities. 

By  law  DOT&E  is  responsible  for  reporting  directly  to  the  Secretary 
of  Defense  and  Congress  on  OT&E  matters.  As  a  member  of  the  DAB,  the 
Director  also  assures  consideration  of  OT&E  findings  for  milestone  decisions. 
To  do  this,  DOT&E  exercise  oversight  responsibilities  for  migor  and  selected 
non-msgor  acquisition  programs.  Among  other  requirements,  the  services 
submit  TEMPs  for  periodic  review,  and  DOT&E  approves  the  OTAs’  test  plans. 
DOT&E  also  reports  to  Congress  on  OT&E  restdts  for  systems  proceeding 
beyond  low  rate  initial  production  (B-LRIP).  Dr  Robert  C.  Dxincan,  who  was 
confirmed  as  Director  in  November  1989,  emphasized  the  need  for  DOT&E  to 
conduct  independent  analysis  of  test  data  and  to  meet  past  concerns  about  the 
quality  and  impartiality  of  OT&E.” 

Reflecting  the  dichotomy  betwemi  OT&E  and  DT&E,  oversight  of  the 
latter  and  management  of  test  resources  belongs  to  the  Under  Secretary  of 
Defense  for  Acquisition’s  Director  of  Test  and  Evaluation.  Prior  to  1992,  the 
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Director  of  Defense  Research  and  Engineering’s  Deputy  Director  for  Test  and 
Evaluation,  abbreviated  as  DDDR&E(T&E},  had  performed  these  functions. 
Until  DOT&E’s  creation,  DDDR&E(T&E)  and  its  predecessors  had  also  been 
responsible  for  oversight  of  OT&E  (except  for  a  brief  period  from  1978  to  1979 
when  OT&E  came  under  the  Assistant  Secretary  for  Program  Analysis  and 
Evaluation  (PA&E). 

Congressional  Influence 

As  previously  indicated,  congressional  interest  in  OT&E  helped  lead 
to  the  establishment  of  independent  OTAs  in  the  early  1970s  and  compelled 
DoD  to  create  DOT&E  a  decade  later.  Much  of  this  interest  is  reflected  in  the 
numerous  studies  and  investigations  performed  by  the  General  Accounting 
Office  (GAO)  as  well  as  frequent  hearings  and  legislative  actions.''* 

A  general  theme  in  statements  by  the  Congress’s  Military  Reform 
Caucus  and  reports  by  the  GAO  has  been  skepticism  about  the  wisdom  of 
concurrent  development,  production,  and  testing  strategies.  Believing  strongly 
in  the  "fly  before  buy"  approach,  they  have  also  reflected  a  philosophy  that 
operational  testing  should  have  more  of  an  adversarial  relationship  to  the 
acquisition  community  and  thereby  help  lead  to  cancellation  of  more  programs. 
Some  congressional  staffers  and  OSD  officials  also  thought  operational  testing 
should  focus  on  demonstrations  and  force-on-force  field  exercises  rather  than 
sophisticated  data  gathering  and  analysis.  The  potential  conflict  of  interest  in 
using  contractor  support  and  data  for  OT&E  has  been  a  particular  concern  of 
Congress,  which  passed  highly  restrictive  provisions  against  this  practice  in 
late  1986."” 

Legislation  passed  in  late  1989  expanded  DOT&E’s  monitoring  role 
by  lowering  the  thresholds  for  certifying  completion  of  OT&E  to  $75  miUion  in 
RDT&E  or  $300  million  in  procurement,  by  requiring  approval  of  test  article 
quantities  for  mqjor  programs  at  Milestone  n  (thereby  tightening  control  over 
LRIP  quantities),  and  by  extending  DOT&E’s  annual  report  to  include  a 
comparison  of  the  TEMP  versus  actual  test  activities.^'  The  new  legislation 
did  not  alleviate  the  restrictions  against  use  of  contractor  personnel  and  data 
that  has  made  combined  DT&E/IOT&E  increasingly  difficult. 

How  to  continue  doing  intensive,  analytical,  yet  efficient  tests  and 
evaluations  of  the  performance  of  increasingly  complex  systems,  and  at  the 
same  time  satisfy  the  political  goals  of  Congress  and  policy  objectives  of  OSD, 
will  undoubtedly  pose  a  continuing  challenge  to  AFOTEC  and  the  Air  Force. 
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Desert  Storm 


The  performance  of  many  "high  tech"  weapons  during  January  and 
Februaxy  1991  in  Operation  Desert  Storm,  although  only  one  factor  contribut¬ 
ing  to  the  coalition’s  decisive  militaiy  victoiy,  helped  put  DoD’s  acquisition  and 
testing  programs  in  a  more  favorable  perspective.  Much  of  the  criticism  and 
bad  publicity  about  operational  testing  during  the  previous  decade  proved 
unfounded  or  exaggerated.  "The  effective  use  of  high  technology  was  a  key 
reason  for  both  the  high  level  of  performance  of  air  and  ground  forces  and  the 
minimization  of  allied  casualties,"  concluded  Rep  Les  Aspin’s  House  Committee 
on  Armed  Services,  which  also  observed  that  "the  decisive  &ctor  in  the  war 
with  Iraq  was  the  air  campaign."^”  DoD’s  Gulf  War  report  also  noted  the 
"extraordinary  effectiveness  of  air  power"  while  concluding  that  "a  revolution¬ 
ary  new  generation  of  high-technology  weapons,  combined  with  innovative  and 
effective  doctrine,  gave  our  forces  the  edge."^  Preliminaiy  findings  on  the 
effectiveness  of  the  weapons  of  Desert  Storm  contrasted  sharply  with  the 
previously  mentioned  reports  of  the  Vietiiam  era. 

AFOTEC  had  tested  at  least  35  of  the  msgor  systems  used  in  Desert 
Storm,  ranging  fiinm  the  A-10  Thimderbolt  n  to  the  Navstar  Global  Positioning 
System.*  An  AFOTEC  study  of  the  wartime  performance  of  six  recently  tested 
systems  showed  significantly  better  operational  effectiveness  in  Desert  Storm 
than  during  OT&E-indicating  both  the  rigor  of  the  operational  tests  as  well 
as  improvements  made  as  the  systems  matured.  Although  less  progress  was 
apparent  in  suitability  factors  (some  of  which  AFOTEC  had  not  been  able  to 
evaluate),^^  the  Center’s  findings  and  similar  data  firom  the  other  OTAs 
confirmed  DOT&E’s  preliminary  assessement  to  Congress  that  "in  general,  the 
experience  of  combat  was  in  keeping  with  the  results  of  OT."*’ 


New  Look  for  Air  Force  OT&E.  1991-1992 

As  described  earlier,  the  creation  of  AFOTEC  in  the  mid-1970s  was 
a  compromise  between  those  who  wanted  a  single  Air  Force  operational  test 
agency  and  those  who  wanted  to  keep  a  decentralized  system.  During  the 
early  1990s,  the  Air  Force  began  another  look  at  its  OT&E  structure  in  the 
midst  of  other  fimdamental  changes. 


*For  a  list  of  the  msgor  systems  tested  by  AFOTEC  firam  1974-1991 
indicating  those  that  participated  in  Desert  Storm,  see  Appendix  C. 
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Changes  in  the  Air  Force 

Redesignation  of  Separate  Operating  Agencies.  On  5  Februazy 
1991,  the  Air  Force  redesignated  most  of  its  Separate  Operating  Agencies 
(SOAs)  and  Direct  Reporting  Units  (DRUs)  as  field  operating  agencies  (FOAs) 
and  assigned  them  to  appropriate  fimctional  chiefs  at  Headquarters  USAF. 
AFOTEC,  which  had  been  an  SOA  since  its  establishment,  became  one  of  only 
three  remaining  DRUs  which,  under  a  new  definition,  reported  directly  to  the 
Chief  of  Staff  of  the  Air  Force  (CSAF).*^  A  factor  that  precluded  AFOTEC 
from  becoming  an  FOA  was  its  Department  of  Defense-mandated  status  as  one 
of  the  services’  independent  operational  test  agencies  (OTAs).*^ 

MAJCOM  Reorganizations.  In  September  1991  the  CSAF,  Gen 
Merrill  A.  McPeak,  and  Secretary  of  the  Air  Force  (SAF),  Dr  Donald  B.  Rice, 
announced  a  sweeping  reorganization  of  the  operational  MAJCOMs.  The  most 
notable  feature  of  this  was  replacement  of  the  Strategic  Air  Command  (SAC), 
Tactical  Air  Command  (TAG),  and  Military  Airlift  Command  (MAC)  by  new  Air 
Combat  and  Air  Mobility  Commands."  This  was  in  addition  to  the  previously 
initiated  replacement  of  Air  Force  Systems  Command  (AFSC)  and  Air  Force 
Logistics  Command  (AFLC)  by  the  Air  Force  Materiel  Command  (AFMC)  and 
formation  of  a  new  Air  Force  Intelligence  Command  (AFIC). 

Creation  of  AF/TE.  In  July  1991  the  Secretary  and  the  Chief  also 
created  a  new  office  in  the  Air  Staff,  the  Director  of  Test  and  Evaluation 
(AF/TE),  to  consolidate  management  over  OT&E  and  DT&E."  AF/TE 
combined  the  functions  previously  performed  by  the  Air  Staffs  OT&E  Division 
(AF/XORT  in  its  final  configuration)  and  Secretariat’s  Directorate  of  T&E 
(SAF/AQV),  which  had  been  created  in  early  1988.  Among  other  initiatives, 
the  new  office  began  a  fresh  look  at  how  the  Air  Force  should  do  OT&E. 
Appointment  of  Lt  Gen  Howard  W.  Leaf,  USAF-Retired  (former  AFOTEC 
commander),  as  the  director  on  28  October  1991  accelerated  this  effort. 

Expansion  of  AFOTEC’s  Mission 

AF/TE*s  Initiative.  In  the  past  decade,  the  decentralized  Air  Force 
OT&E  structure  had  grown  increasingly  out  of  step  with  the  rest  of  the 
Department  of  Defense-whose  directives  had  long  called  for  each  of  the 
military  services  to  have  a  single  operational  test  agency  independent  of 
developing  and  using  commands."  As  noted  earlier,  the  Navy  had  chartered 
sudi  an  organization  in  1971,  and  the  Army  fully  centralized  its  operational 
testing  and  evaluation  under  a  new  command  in  1990.  General  Leaf  was  not 
satisfied  with  the  Air  Force’s  existing  OT&E  structure,  and  with  the  support 
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of  AFOTEC’s  Commander,  Maj  Gen  Marcus  A.  Anderson,  included  a  proposal 
to  centralize  responsibility  for  most  tests  as  one  of  his  most  important  initia¬ 
tives  to  improve  Air  Force  T&E."  In  December  1991  General  Leaf  obtained 
approval  firom  the  SAF  and  CSAF  for  the  expansion  of  AFOTEC’s  mission  to 
include  all  of  the  initital  and  qualification  testing  that  had  been  conducted  by 
the  MAJCOMs.  The  goals  for  this  restructure  included  improved  efficiency, 
impartiality,  and  credibility.*” 

Implementing  the  Realignment.  A  26  December  1991  announce¬ 
ment  by  Gen  Michael  P.C.  Gams,  the  Air  Force  Vice  Chief  of  Staff,  launched 
this  most  significant  change  in  the  Air  Force’s  OT&E  mission  since  the 
establishment  of  AFTEC  in  1974.  His  message  directed  that  AFOTEC  assume 
responsibility  for  all  initial  and  qualification  OT&E  plus  selected  FOT&E.”” 

For  AFOTEC,  the  new  year  brought  the  abmpt  challenge  of  planning, 
negotiating,  programming  and  implementing  this  realignment  by  1  June  1992- 
the  same  date  that  the  new  Air  Mobility  and  Air  Combat  Commands  would 
stand  up.  The  Center  assembled  a  transition  team,  which  worked  closely  with 
representatives  firom  the  affected  MAJCOMs  to  review  each  of  several  hundred 
existing  test  programs.  After  scrubbing  this  list,  they  determined  which  were 
appropriate  for  transfer  to  AFOTEC,  which  should  be  completed  by  existing 
test  teams,  and  which  were  eligible  for  continued  MAJCOM  management” 

Once  AFOTEC  and  the  commands  agreed  on  the  status  of  test 
programs,  they  had  to  determine  how  mudi  funding,  how  many  positions,  and 
which  people  would  be  transferred  to  the  Center.”  As  in  any  mqjor  realign¬ 
ment,  such  actions  were  sometimes  painful  for  the  losing  organizations.  Yet 
a  spirit  of  cooperation  generally  prevailed,  and  most  of  the  details  were  worked 
out  in  a  mere  five  months. 

Program  Transfers.  On  4  May  1992,  General  Anderson  briefed  the 
Air  Force  Chief  of  Staff  on  the  progress  to  date.”  Vfith  the  CSAFs  approval, 
the  transfers  took  place  as  scheduled.  On  1  June  the  number  of  programs 
being  conducted  by  AFOTEC  increased  fi:t>m  41  to  186.  These  included  those 
whidi  had  been  tiie  responsibility  of  AFCCs  Technical  Integration  Center, 
whMe  mtire  OT&E  Directorate  (TICyrE)  was  reborn  as  AFOTEC  Det  1  on 
that  date.  Fifty-nine  additional  tests,  already  underway,  were  "grandfathered" 
for  the  MAJCOMs  to  finish.” 

After  meeting  the  June  target  date,  AFOTEC  and  the  MAJCOMs 
wrapped  up  most  of  the  remaining  details  by  30  September  1992.  Fifty 
additional  MAJCOM  test  programs  were  identified  during  the  summer  for 
transfer  to  AFOTEC,  bringing  the  total  to  195  (with  over  a  hundred  more  that 
mi|^t  also  require  involvement  by  the  Center  in  the  future).  To  meet  its 
increased  workload,  AFOTEC  received  more  than  200  new  positions,  which 
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with  previously  programmed  losses,  brought  its  authorized  strength  to  just 
over  800.  The  main  challenge  involved  previously  unidentified  tests  and 
fimding  requirements  that  forced  AFOTEC  to  prioritize  test  programs  to  deter¬ 
mine  how  best  to  apply  the  limited  money  available.”  Obviously,  many  chal¬ 
lenges  lay  ahead  for  AFOTEC  in  managing  its  expanded  mission  at  a  time  of 
nugor  fiscal  pressures. 

Changes  in  AFOTEC’s  Organization 

In  harmony  with  the  restructure  of  Air  Force  OT&E,  AFOTEC 
simultaneously  executed  the  most  thorough  reorganization  in  its  history. 
Announced  by  General  Anderson  on  18  February  1992,  it  became  effective  on 
18  May.  The  reorganization  had  several  complementary  objectives,  all 
consistent  with  the  principles  of  total  quality  management  and  its  goals  of 
empowering  teams  at  the  working  level  and  meeting  customer  expectations  for 
fast  and  efficient  service. 

1974-1991.  Prior  to  1992,  AFOTEC’s  organization  followed  a  matrix 
structure  in  which  each  OT&E  program  drew  upon  a  variety  of  personnel  from 
functional  areas  in  other  directorates.  Most  test  managers,  who  had  responsi¬ 
bility  for  planning  and  coordinating  individual  OT&E  programs,  were  assigned 
to  one  of  several  divisions  in  the  Directorate  of  Test  and  Evaluation  (TE).  The 
director  of  TE  also  supervised  detachments  and  test  teams  in  the  field. 
Starting  in  1977,  the  Directorate  of  Plans  and  Resources  (XR)  began  to  perform 
advance  planning  for  many  OT&E  programs  with  its  own  test  managers.  In 
1981,  XR  split  into  the  Directorates  of  Plans  and  PoUqt  (XP)  and  Resources 
and  Support  (RM).  For  technical  and  support  expertise  on  their  test  support 
groups,  TE’s  or  XP’s  test  managers  relied  on  representatives  firom  other 
directorates  and  offices.  The  TSGs  included  members  firom  the  Directorate  of 
Analysis  (OA),  the  Directorate  of  Logistics  (LG),  as  well  as  RM  and  special 
staff  or  liaison  offices  for  safety,  security,  weather,  and  intelligence  expertise. 
AFOTEC  managed  joint  tests  in  a  Directorate  of  Special  Test  (JT),  which 
because  of  a  dedining  JT&E  workload,  became  an  XP  division  in  1987.  Also 
in  1987,  the  Center's  growing  participation  in  highly  classified  programs  with 
special  access  requirements  led  to  formation  of  the  Directorate  of  Special  Test 
(ST).  In  1990  a  provisional  organization  und^  an  assistant  for  range  matters 
(formed  in  1989)  became  the  Directorate  of  Test  Capability  (CR)  to  provide 
expertise  on  threat  capabilities,  range  resomxes,  and  related  matters.  Figure 
3  shows  the  final  version  of  AFOTEC’s  matrix  organization  prior  to  the 
reorganization  of  1992. 
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Figure  8.  Organizatioii  of  AFOIEC*  1991 


The  matrix-type  organizationt  whidi  was  the  sulgect  of  nuyor  studies 
published  in  1981  and  1987,  was  an  efficient  and  flexible  way  to  use  special¬ 
ized  personnel  to  support  dianging  program  workloads  while  keepng  subject 
matter  exparts  together.  But  this  type  of  structure  did  not  reflect  the  mission 
areas  of  the  systems  tested.  It  featured  vertical  divisions  that  split  TSG 
mmnbers  among  difiinrent  offices  and,  along  with  horizontal  layering  of 
directorates  into  divisions  and  brandies,  fostered  a  lengthy  coordination 
process.  It  also  gave  the  Director  of  TE,  who  was  responsible  for  six  large 
divisions  as  well  as  detachments  in  the  Add,  a  disproportionate  q>an  of 
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control.**  As  shown  in  Figure  4,  there  were  some  changes  to  the  directorates 
over  the  years,  but  the  fundamental  structure  of  1974  remained  intact  until 


1992. 
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Figure  4.  Evolution  of  M^Jor  AFOlEC  Directorates,  1974-1992 


Reorganization  of  1992.  The  most  visible  feature  of  the  reorgani¬ 
zation  in  the  headquarters  was  the  realignment  of  nugor  directorates.  As 
shown  in  Figure  4,  the  Directorate  of  Test  and  Evaluation  (TE),  previously 
responsible  for  all  but  a  small  number  of  special  programs,  gave  birth  to  three 
new  T*  directorates  organized  by  mission  areas;  Weapon  Systems  (TF),  C*I 
Systems  (TK),  and  Space  and  Missile  Systems  (TS).  As  in  the  existing 
Directorate  of  Special  Test  (SD,  each  now  included  its  own  analysts.  These 
were  reassigned  from  the  former  Directorates  of  Analysis  (OA)  and  Logistics 
(LQ),  whose  remaining  functions  were  consolidated  into  a  new  Directorate  of 
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Systems  Analysis  (SA).  Likewise,  the  Directorates  of  Plans  and  Policy  (XP) 
and  Test  Capability  (CR)  were  consolidated  into  the  Directorate  of  Plans, 
Policy,  and  Reqmrements  (XR).  The  other  large  directorate.  Resource 
Management  (RM),  remained  largely  unaffected  by  the  reorganization.^  The 
organization  chart  below  (Figure  5)  depicts  the  msgor  features  of  AFOTEC’s 
structure  after  18  May  1992. 
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Figure  6.  Organization  of  AFOTEC,  1992 


In  addition  to  realigning  directorates,  the  reorganization  eliminated 
an  entire  bureaucratic  layer  of  middle  management  by  blending  what  had  been 
divisions  and  brandies  into  teams.  Tlw  new  "flatter”  structure  improved 
communications  flow  and  pushed  auUunrity  down  lower  in  the  organization. 
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This  reinforced  the  status  of  test  managers  and  focused  attention  on  their  test 
support  groups,  which  remained  the  key  elements  for  planning,  coordinating, 
and  analyzing  tests.  Instead  of  being  formed  by  specialists  who  were 
"matrixed"  fivm  various  directorates,  TSG  members  now  came  from  the  same 
dosely-knit  teams.  To  help  build  an  initial  test  concept  for  many  programs, 
XR  formed  a  new  test  concept  group  (TCG)  of  highly  experienced  personnel. 

The  reorganization  also  strengthened  the  role  of  AFOTEC’s  detach¬ 
ments  in  the  field,  which  were  expanding  in  size  and  importance  as  they  took 
over  many  of  the  former  MAJCOM  tests.  Detachment  commanders,  who 
previously  came  under  the  TE  Director,  now  reported  directly  to  the  AFOTEC 
Commander.  The  key  role  of  test  teams  in  executing  tests  was  also  enhanced. 
The  Headquarters  would  continue  to  plan  tests,  while  the  dets  and  OLs  would 
execute  them,  with  the  transfer  of  responsibility  now  occurring  at  test 
readiness  briefings  (approximately  30  days  prior  to  testing). 

The  intent  of  the  reorganization  was  improved  planning  and  support 
for  AFOTEC’s  test  teams  in  the  field— and  better,  more  timely  OT&E 
products.”  As  a  result  of  the  program  transfers  and  AFOTEC  reorganization, 
the  Air  Force  now  had  a  single,  expanded  operational  test  agenpy  to  assure 
independence  and  to  help  improve  the  credibilily  of  its  OT&E  mission. 


Conclusion 


Alter  the  restructure  of  Air  Force  OT&E  and  its  internal  reorgani¬ 
zation,  AFOTEC  stood  poised  to  take  on  the  challenges  of  the  post-Cold  War 
era.  Although  the  Air  Force  appeared  certain  to  continue  shrinking  in  size, 
and  its  budget  to  remain  under  severe  pressures,  there  seemed  to  be  a  consen¬ 
sus  on  the  need  to  maintain  the  technological  advantages  recently  demon¬ 
strated  in  Desert  Storm.  Even  with  the  cancellation  of  many  acquisition 
programs,  operational  testing  woiild  be  required  on  those  new  systems  that 
remained  under  consideration  for  production.  Tests  and  assessments  would 
also  be  needed  for  modifications  to  existing  equipment  and  prototypes  of 
systems  that-under  DoD^s  new  acquisition  strategy-might  be  developed  but 
not  produced  in  quantity.  AFOTEGs  chief  diallenge  would  be  to  find 
innovative  ways  to  test  this  wide  range  of  systems  at  the  best  possible  cost 
while  continuing  to  assure  their  operational  effectiveness  and  suitability.” 
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30.  92nd  Cong~lst  Sess,  Public  Law  92-156  (85  Stat  423),  17  Nov  71,  p  463. 

31.  Report  of  the  Commission  on  Goveminent  Procurement  dted  above,  Vol 
II,  p  164. 

32.  St  Louis,  Toli(y  Affecting  OT&E,"  pp  17-20. 

33.  Hist  of  the  Dep  Dir  for  OT&E  (AFOCOOW),  Jan-Jun  71. 

34.  AFR  80-14,  "Test  and  Evaluation,"  12  May  72. 

35.  Hist  of  AF/XOOW,  Jan-Jim  73. 

36.  US  General  Accounting  Office,  "Statement  of  Elmer  B.  Staats,  Comptroller 
General  of  the  United  States,  before  the  Committee  on  Armed  Services,  House 
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AFOTEC,  1991.  AFOTECR  55-9,  "Management  of  Operational  Test  and 
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48.  See  Special  Study  (FOUO),  Benson,  "Development  of  the  Electronic 
Combat  Test  Capability  (ECTC)  Program,  1984-1989,"  (AFOTEC/RS,  Jul  89) 
and  Hist  of  AFOTEC,  1990,  pp  54-57. 
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Autumn  1971,  pp  4-30;  Capt  Rich  Reitan,  "TAC  OT&E,"  USAF  Fighter 
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meetings  and  discussions  witib  AFOTEC  officials,  1986-1992. 
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64.  For  a  snapshot  of  OPTEVFOR  in  the  Vietnam  era,  see  RADM  Marshall  E. 
Domin,  T^avd  Weapons  Evaluation,"  Ordnance.  Mar-Apr  67,  pp  474-77. 
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72.  OTD  Guide  cited  above,  pp  2-16-2-25, 12-10. 
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76.  For  example,  see  GAO/NSIAD  Report  87-57,  Operational  Test  and 
Evaluation  Can  Contribute  More  to  Deeimonmaking.  Dec  86,  Appendix  I, 
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84.  See  Hist  of  AFOTEC,  1990,  p  28. 

85.  Memo,  SAF/PA,  "Air  Force  Association  Speech  by  Air  Force  Secretary 
Rice,"  17  Sep  91,  w/atch  text  of  speedi  and  White  Paper,  "Air  Force  Restruc¬ 
ture;"  James  W.  Canan,  "Convention  91:  The  ’Back  to  Basics’  Plan,"  Air  Force 
Magazine.  Nov  91,  pp  82-85. 

86.  Memo,  (^en  Merrill  A.  MePeak  and  Secretary  Donald  B.  Rice  to  SAF/AQ 
et  al,  "Air  Force  Test  and  Evaluation-Action  Memorandum,"  19  Jul  91,  SD  I- 
45  in  Hist  of  AFOTEC,  1991. 

87.  Most  recently  stated  in  DoDD  5000.1,  "Defense  Acquisition,"  23  Feb  91, 

pI-8. 

88.  See  Hist  of  AFOTEC,  1991,  pp  28-29,  32-39. 

89.  Oral  History  Interview,  Benson,  AFOTEC/RS,  with  Lt  Gen  Howard  W. 
Leaf,  USAF,  Ret.,  8  and  10  Sep  92;  Brfg,  AF/TE,  "Air  Force  Test  and 
Evaluation  ^^alignment:  Stepping  up  to  the  Need  for  Change,"  24  Feb  92. 

90.  Msg,  HQ  USAF/CV  to  ALMAJCOM-SOA,  "USAF  lOT&E  and  QOT&E 
Responsibility  Realignment,"  261200Z  Dec  91,  SD  1-69  in  Hist  of  AFOTEC 
1991. 

91.  Intvws,  Benson  with  Col  Clarke  D.  Hubbard,  Special  Assistant  to  the 
Commander  for  the  OT&E  Restructure  (AFOTEC/CCS),  Aug  92. 
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92.  See  HQ  AFOTEC  Programming  Plan  (PPlan)  92-1  (FOUO),  ‘Transfer  of 
Initial...Qualification  Operational  Test  and  Evaluation,"  30  Apr  92,  and  HQ 
USAF  Program  Action  Directive  92-09  (FOUO),  "Transfer  of  Msgor  Clommand 
Initial  and  Qualification  Operational  Test  to  [AFOTEC],"  15  May  92,  for  details 
(to  be  SDs  in  Hist  of  AFOTEC  1992). 

93.  Brfg,  Meg  Gen  Marcus  A.  Anderson,  AFOTEC/CC,  and  Col  Clarke  D. 
Hubbard,  AFOTEC/CCS,  to  CSAF,  "Realignment  of  Air  Force  Operational  Test 
and  Evaluation,"  4  May  92  (to  be  SD  in  Hist  of  AFOTEC,  1992). 

94.  AFOTEC  Blanket  Work  Directive  92-01,  Trograms  that  have  been 
transferred,"  17  Jun  92  (to  be  SD  in  Hist  of  AFOTEC  1992). 

95.  Brfgs  by  AFOTEC/RM  on  fimdingfi^m  late  Sep  through  mid-Nov  92;  Ltr, 
AFOTECVCCE  to  CV  et  al,  "Offsite  brutes,"  28  Oct  92;  Brfg,  "Implementation 
of  OT&E  Realignment,"  2  Nov  92. 

96.  Col  David  T.  Archino,  later  AFOTEC  Chief  of  Staff,  chaired  the  following 
two  studies:  Report  on  the  AFOTEC  Mission,  May  81,  and  Review  of  the 
AFTE(j  Organization,"  Jun  81,  SDs  1-23  and  1-28  in  Hist  of  AFTEC  1981.  A 
member  of  the  USAF  Academes  Dept  of  Behavioral  Sdences  and  Leadership 
led  a  3-person  team  of  consultants  t^t  issued  an  Organizational  Assessment 
of  AFOTEC  in  Sep  87,  SD  1-99,  Hist  of  AFOTEC  1987. 

97.  HQ  AFOTEC  PPlan  (FOUO),  "Internal  Reorganization  of  AFOTEC,"  22 
May  92,  no  FOUO  info  used. 

98.  Mig  Gen  Marcus  A.  Anderson,  "AFOTEC:  Increased  Responsibility  and 
Organizational  Improvement,"  TIG  Brief.  No.  5,  Sep-Oct  92;  Brfg,  AFOTEC/- 
CC,  "Internal  AFOTEC  Realignment,"  2  Nov  92. 

99.  Videotape  (FOUO)  AFOTEG/RSV,  Address  to  AFOTEC  by  Gen  Michael 
P.C.  Cams,  Vice  CSAF,  2  Nov  92,  no  FOUO  info  used. 
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Appendix  A 

SELECTED  BIBUOGRAPHY  OF  REPORTS  AND  STUDIES 

The  following  includes  papers  prepared  for  the  Air  War  College  (AWC),  Air  Command 
and  Staff  College  (ACSC),  Defense  Systems  Management  College  (DSMC),  and  the 
National  War  College  (NWC>-most  of  which  are  also  available  from  the  Defense 
Technical  Information  Center. 

AFTEC/AFOTEC  Histories,  1974-1992,  with  supporting  documents 

Benson,  L.R.,  Cofer,  RJ.,  and  lioyd,  C.D.,  "An  Illustrated  History  of  AFOTEC," 
Preliminary  Edition,  AFC/TEC/RS,  Nov  89 

Benson,  LR,  Janson,  C.R,  and  Saluda,  SA.,  "Test  Limitations:  Experience  of  the  Air 
Force  Operational  Test  and  Evaliiation  Center,  1974-1990,"  AFOTEC/RS,  Jun  91. 

Bryden,  ^^^am  D.  Jr,  M^j,  USAF,  "The  Test  and  Evaluation  Evolution  (Relative  to 
the  US  Air  Force),"  DSMC  Study  Report  PMC  74-1,  May  74 

Dolan,  IK^lUam  G.,  Lt  Col,  USAF,  "Operational  Test  and  Evaluation:  The  Key  to 
Credibility,"  AWC  Research  Report  4896,  Apr  73 

Everly,  Keith  W.,  Mq,  USAF,  "United  States  Air  Force  Policy  for  Operational  Test 
and  Evaluation,"  ACSC  Studcmt  Report  87-0800,  Apr  87 

Fallon,  Clifford  B.,  Mig,  USAF,  "Operational  Test  and  Evaluation  Planning 
Procedxires,"  ACSC  Research  Study  0930-74,  May  74 

Guild,  Richard  E.,  Col,  USAF,  "Implications  of  Legislation  Regarding  Operational 
Testing  and  Evaluation,"  NWC  Strategic  Study,  May  84 

Headquarters  USAF,  DCS/Plans  and  Operations,  semiannual  historical  reports,  1964- 
1974 

Hersman,  Walter  C.,  Msg,  USAF,  "Operational  Test  and  Evaluation  in  the  Acquisition 
of  USAF  Weapon-Systems,"  ACSC  foch  Study  1270-72,  May  72 

Hoblit,  Jerry  N.,  Lt  Col,  USAF,  "Test  and  Evaluation:  The  Persistent  Problem,"  AWC 
Research  Report  372,  Apr  78 

Holmes,  James  L.,  Lt  Col,  USAF,  "Problems  and  Recommendations  Concerning  Test 
and  Evaluation  Policy,"  AWC  Research  Report  376,  Apr  78 

Hoover,  Ronald  C.,  Msg,  USAF,  "Proposals  for  Improved  Management  of  Air  Force 
Test  axid  Evaluation,"  ACSC  Report  1095-80,  May  80 
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Kessler,  David  C.,  Lt  Col,  USAF,  "A  Study  of  USAF  Test  and  Evaluation  Policy," 
DSMC  Study  Project  Report  PMC  77-2,  Nov  77 

Long,  Christoper  S.,  Mcy,  USAF,  "Operational  Test  and  Evaluation  in  the  Weapon 
System  Acquisition  Proce^,"  ACSC  Student  Rpt  82-1550,  Mar  82 

Oertel,  Robert  E.,  Mqj,  USAF,  "Operational  Test  and  Evaluation:  The  Quest  for 
Independence,"  R^ean^  Report  AU  ARI-85-8,  Dec  85 

Randolph,  Bernard  P.,  Lt  Col,  USAF,  "Organizing  for  Operational  Test  and 
.Evaluation,"  AWC  Professional  Study  5508,  Apr  74 

St  Louis,  Terrence  R,  AFOTEC/HO,  "The  Evolution  of  Policy  Affecting  Air  Force 
Operational  Test  and  Evaliiation,  1909-1981,"  Sep  83 

Van  Pelt,  Larry  G.,  Col,  USAF,  Tlight  Test  Concept  Evolution,"  AWC  Research 
Report  120-81,  Apr  81 
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COMMANDERS  AND  VICE  COMMANDERS  OF  AFOTEC 
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Commanders: 


Mq  Gen  John  J.  Bums 
Mig  Gen  Richard  G.  Cross,  Jr. 
Col  Stephen  E.  Moore 
Mig  Gen  Robert  A.  Rushworth 
M{g  Gen  Howard  W.  Leaf 
Msg*  Gen  Wayne  E.  Whitlatch 
Msg*  Gen  Richard  W.  Phillips,  Jr. 
Msg  Gen  Michael  D.  Hall 
Mig  Gen  Cecil  W.  Powell 
Msg*  Gen  Peter  D.  Robinson 
Msg*  Gen  Marcus  A  Anderson 

\^ce  Commanders: 

Col  Harold  K.  Wimberley 
Col  Stephen  E.  Moore 
Col  Hervey  S.  Stockman 
Col  Charles  H.  Hausenfleck 
Col  Paul  N.  Chase 
Col  Ralph  F.  Wetzl 
Col  Jon  L  Lucas 
Col  Joseph  E.  Menick 
Col  Robert  A  Heston 
Col  John  A  Judd 


1  January  •  25  August  1974 

26  August  1974  •  31  August  1975 

1  September  1975  ■  9  November  1975 
10  November  1975  -  30  September  1976 
1  October  1976  •  31  May  1980 
1  June  1980  •  26  May  1982 

27  May  1982  •  29  August  1985 
30  August  1985  -  30  June  1987 
1  July  1987  - 18  January  1990 
19  January  1990  -  18  July  1991 
19  July  1991  - 


5  July  1974  •  31  August  1975 

10  November  1975  -  31  April  1977 
1  May  1977  •  28  August  1977 
29  August  1977  •  1  November  1982 

6  Jsuitisury  1983  •  19  June  1984 
20  June  1984  •  3  July  1985 

3  July  1985  •  14  October  1987 
15  October  1987  •  28  June  1989 
29  June  1989  •  31  August  1992 
1  September  1992  • 
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SYSTEMS  TESTED  BY  AFOTEC,  1974-1992 
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The  following  list  shows  various  weapons  and  other  systems  tested  and  evaluated  by 
AFOTEC  prior  to  the  program  transfers  of  1992  and  the  inclusive  dates  of  testing. 
Many  of  the  entries  cover  miiltiple  OT&E  programs,  with  various  tests  (e.g.,  lOT&Es, 
FOT&Es,  OUEs)  performed  during  the  time-spans  indicated.  It  does  not  include 
operational  assessments  (OAs  or  EOAs)  of  s j^tems  still  in  development.  Systems  used 
in  or  deployed  to  Operation  Desert  Storm  are  indicated  by  an  asterisk  (*). 


A- 10  Thunderbolt  11, 1974-77* 

A-10  LANTIRN,  1982-83 
A- 10  Operation^  Flight  Trainer 
(OFT),  1980-82 

Advanced  Aerial  Refueling  Boom, 
1977-78* 

Advanced  Airborne  Command  Post 
(E-4B),  1977-79 

Advanced  Medium  Range  Air-to-Air 
MissUe  (AMRAAM),  1981-  * 
Advanced  Synthetic  Aperture  Radar 
System  (ASARS-2),  1982* 

AGM-65  Maverick  (varioiis  models), 
1974-85* 

AGM-86B  Air  Launched  Cruise  Mis¬ 
sile  (ALCM),  1979-81* 

AGM-88  High  Speed  Anti-Radiation 
Missile  (HARM),  1979-87* 
AGM-109  ALCM,  1979-80 
AGM-122  SideARM,  1983-84 
AGM-130,  1989-90 
A1M-7F/M  Sparrow,  1975-82* 
AIM-9iyM  Sidewinder,  1975-81* 
AIM-120  -  See  AMRAAM* 

Airborne  Self  Protection  Jammer 
(ASPJ/ALQ-165),  1988-89 
Airborne  Warning  and  Control  Sys¬ 
tem  (AWACS/E-3A),  1974-82* 

Air  Force  Satellite  Communications 
(AFSATCOM),  1975-82* 
AN/ALR-56M  Radar  Warning 
Receiver,  1988 

AN/ALR-74  Radar  Warning  Receiver, 
1984-88 

AN/AFR-38  (F-4G),  1986 


Anti-Satellite  (ASAT)  air-launched 
missile,  1983-88 
AQM-81A  Firebolt,  1984-85 
Automated  Data  Processing  System, 
1979-83 

Automated  Remote  Tracking  System 
(ARTS),  1988-89* 

Automated  Technical  Control  Pro¬ 
gram  (ATEC),  1977-78 
B-l/B-lA,  1974-81 
B-IB  Lancer,  1984-91 
B-IB  Weapon  System  Trainer,  1991- 
B-52  (various  models  and  systems), 
1977-86* 

B-52  Weapon  System  Trainer  (WST), 
1981-84 

BGM-109  Ground  Launched  Cruise 
MissUe  (GLCM),  1982-84 
BQM-34F  Target  Drone,  1975 
C-5A  Galaxy,  1980-81* 
C-141B/yC-141B  Starlifter,  1977-80* 
CBU-89/B  Gator,  1980-82* 

Cobra  Dane,  1976-77 
Common  Strategic  Rotary  Laimcher, 
1985-86 

Clommunications  Data  Link  Jammer 
(CDLJ),  1982 

Consolidated  Space  Operations 
Center  (CSOC),  1989-  * 

Cruise  MissUe  Defense,  1985-87 
DSU-16/B  Target  Detector,  1976-77 
Defense  Support  Program  (various 
subsystems),  1978-86* 
E-3A-SeeAWACS* 

E-4B  -  See  AACP 


48 


EF-lllA  Raven,  1977-82* 

EP-lllA  OFT,  1985-87 
F-4G  Advanced  Wild  Weasel, 
1975-78* 

F-5E/F,  1975-76  (RSAF*) 

F-15  Eagle,  1975-77* 

F-15E  Strike  Eagle,  1988-90* 
F-16/YF-16  Falcon,  1974-90* 

F-16  OFT/WST,  1981-89 
GBU-15  GUde  Bomb,  1983-87* 

GLCM  -  See  BGM-109 
Ground  Wave  Emergency  Network 
(GWEN),  1984-88 
HH-60  Combat  Helicopter,  1984-86 
Joint  Tactical  Fusion,  1984 
Joint  Tactical  Information  Distribu¬ 
tion  System  (JTIDS),  various 
components,  1978-89* 

Joint  Surveillance  and  Target  Attack 
Radar  System  (JSTARS),  1990-  * 
KC-10  Tanker,  1980-81* 

KC-135  WST,  1981 
KC-135R,  1982-84* 

Low  Altitude  Navigation  and  Target¬ 
ing  Infrared  for  Nis^t  (LANTIRN), 
1982-87* 

Low  Level  Laser  Gmded  Bomb 
(GBU-22/23),  1982-85* 

Milstar,  1984- 

Modular  Control  Equipment  (MCE), 
1986-87* 

NATO  Airborne  Early  Warning 
Ground  Enviroxunent  Integration 
Segment  (NAEGIS),  1982-83 
Navstar  Global  Positioning  System 
(Space,  Control,  and  User  Eqmp- 
ment  Segments),  1980-  * 

Next  Generation  Weather  Radar 
(NEXRAD),  1986-89 
North  Warning  System,  1989 
Over-the-Horizon  Backscatter  (OTH- 
B)  Radar,  1981-1991 
Pave  Mover/Assault  Breaker, 

1981- 82 

Peacekeeper  in  Minuteman  Silos, 

1982- 89 

Piper  Enforcer  (PA-48),  1984 
Pndsion  Location  Strike  System 


(PLSS),  1986-87 

Remote  Controlled  Tactical  Airborne 
SIGINT  System,  1983-84 
Seek  Comm,  1982 
Seek  Talk,  1981 
Sensor  Fuzed  Weapon  (SFW), 
1990-91 

Short-Range  Attack  Missile  (SRAM) 
-n,  1990-91 

Space  Defense  Operations  Center 
(SPADOC),  1986-87 
Space  Transportation  System  (STS), 
various  components,  1980-87 
Strategic  Air  Command  Digital  Info 
Network  (SACDIN),  1984-86 
T-46A  Next  Generation  Trainer, 
1985-87 

Tadt  Rainbow  (AGM-136),  1989-91 
Tactical  Ground  Intercept  Facility  n, 
1983-84 

TR-1  Tactical  Reconnaissance  Sys¬ 
tem,  1986* 

TRI-TAC  (numerous  components), 
1979-87* 

YC-14  Transport,  1976-77 
YC-15  Transport,  1975-77- 
YF-17  lightweight  Fighter,  1974-75 


*Used  in  Desert  Stem 


49 


GLOSSARY  OF  ABBREVIATIONS  AND  ACRONYMS 
(Does  not  include  weapon  system  designations  identified  in  Appendix  C) 


AAF 

Army  Air  Forces 

DMR 

Defense  Management  Report 

ACC 

Air  Combat  Command 

DoD 

Department  of  Defense 

AF 

Air  Force 

DOT&E 

Director  of  Operational 

AFCC 

AF  Communications 

Test  and  Evaluation 

Command 

DSARC 

Defense  System  Acquisition 

AFIC 

AF  Intelligence  Command 

Review  Council 

AFFTC 

AF  Flight  Test  Center 

DT&E 

Development  Test  and  Evalu¬ 

AFM 

AF  Manual 

ation 

AFMC 

AF  Materiel  Command 

ECTC 

Electronic  Combat  Test 

AFOTEC 

AF  Operational  Test  and 

Capability 

Evaluation  Center 

EOA 

Early  Operational  Assessment 

AFOTECR 

AFOTEC  Regulation 

ESC 

Electronic  Security  Command 

AFPD 

AF  Policy  Directive 

FOA 

Field  Operating  Agen^ 

AFR 

AF  Regulation 

FOT&E 

Follow-on  Operational  Test 

AFSARC 

AF  Systems  Acquisition 

and  Evaluation 

Review  Council 

FSD 

Full  Scale  Development 

AFSC 

AF  Systems  Command 

GAO 

General  Accounting  Office 

AFTEC 

AF  T&E  Center 

ICBM 

Intercontinental  Ballistic 

AFSPACE- 

Missile 

COM 

AF  Space  Command 

IDA 

Institute  for  Defense  Analfyses 

ALCENT 

Airlift  Center 

lEP 

Independent  Evaluation 

AMC 

Air  Mol^ity  Command 

Plan 

APGC 

Air  Proving  Ground  Command 

IOC 

Initial  Operational  Capability 

AQV 

Director  for  T&E,  Assistant 

lOT&E 

Initial  Operational  Test 

SAF  for  Acquisition 

and  Evaluation 

ARDC 

Air  Research  and  Development 

JRMET 

Joint  Reliability  and  Main¬ 

Command 

tainability  Evaluation  Team 

ATC 

Air  Training  Command 

JCS 

Joint  Chiefs  of  Staff 

B-LRIP 

Beyond  LRIP 

JT&E 

Joint  Test  and  Evaluation 

C*E 

Continuous  Comprehensive 

LFT 

Live  Fire  Testing 

Evaluation 

LG 

Director  of  Logistics 

CNO 

Chief  of  Naval  Operations 

LRIP 

Low  Rate  Initial  Production 

COEA 

Cost  &  Operational  EJffective* 

MAC 

Militaiy  Airlift  Command 

ness  Analysis 

MAJCOM 

Mqjor  Command 

COI 

Critical  Operational  Issue 

MCOTEA 

Marine  Corps  Operational 

COR 

Continental  Operating  Range 

Test  and  Evaluation  Activity 

COTS 

Contractor  off-the-shelf 

MOA 

Memorandum  of  Agreement 

CTP 

Combined  Test  Force 

MOE 

Measure  of  Effectiveness 

CR 

Director  of  Test  Capability 

MOT&E 

Multiservioe  Operational 

DAB 

Defense  Acquisition  Board 

Test  &  Evaluation 

DCS 

Deputy  Chief  of  Staff 

MST&E 

Multiservice  Test  &  Eval¬ 

DDR&E 

Director  Defense  Research 

uation 

and  Engineering 

NDI 

Nondevelopmental  Item 

DDDR&E 

NORAD 

North  American  Air  Defense 

(T&E) 

Deputy  DDR&E  for  T&E 

Command 

DEMVAL 

Demonstration  &  Validation 

OA 

Director  of  Analysis 

Det 

Detadiment 

OA 

Operational  Assessment 
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OL 

Operating  Location 

SAR 

Special  Access  Required 

OPEVAL 

Operational  Evaluation 

SMOTEC 

Special  Missions  OT&E 

OPTEC 

Operational  Test  and  Evalu¬ 

Center 

ation  Command 

SON 

Statement  of  Operational 

OPTEV- 

Need 

FOR 

Operational  Test  and  Eval¬ 

SPO 

Systems  Program  Office 

uation  Force 

ST 

Director  of  Special  Test 

ORD 

Operational  Requirements 

TAC 

Tactical  Air  Command 

Document 

TAF 

Tactical  Air  Forces 

ORMAS 

Operational  Resources  Man¬ 

TAWC 

Tactical  Air  Warfare  Center 

agement  Assessment  System 

T&E 

Test  and  Evaluation 

OSD 

Office  of  the  Secretary  of 

TE 

Director  of  Test  &  Ehraluation 

Defense 

TECHEVAL 

Technical  Evaluation 

or 

Operational  Test 

TECO 

Test  &  Evaluation  Coordina¬ 

OTA 

Operational  Test  Agency 

tion  Officer 

OT&E 

Operational  Test  and  Evalu¬ 

TEC 

TEXCOM  Ehcperimentation 

ation 

Center 

OTD 

Operational  Test  Director 

TEMP 

T&E  Master  Plan 

OTEA 

Operational  Test  and  Evalu¬ 

TEXCOM 

Test  and  Experimentation 

ation  Agency 

Command  (OPTEC) 

OTEC 

OT&E  Center  (AFCC) 

TF 

Director  of  Weapon  Systems 

OTPO 

Operational  Test  Project 

TFWC 

Tactical  Fighter  Weapons 

Officer 

Center 

OTSA 

OPTEC  Threat  Support 

TIC 

Technical  Integration  Center 

Activity 

TK 

Director  of  C4I  Systems 

OUE 

Operational  Utility  Evalu¬ 

TPR 

Test  Planning  Review 

ation 

TRADOC 

Training  and  Doctrine  Com¬ 

PACAF 

Pacific  Air  Forces 

mand 

PEO 

Program  Executive  Office 

TS 

Director  of  Space  &  Missile 

PMD 

Program  Management  Direc¬ 

Systems 

tive 

TSG 

Test  Support  Group 

PPQT 

Pre-Production  Qualification 

USAFE 

United  States  Air  Forces  in 

Test 

Europe 

PQT 

Production  Qualification  Test 

USD(A) 

Undersecretary  of  Defense 

PQDR 

Product  Quality  Deficient 

(Acquisition) 

Report 

WSEP 

Weapon  System  Evaluation 

QOT&E 

Qualification  Operational 

Test  and  Evaluation 

XOORE 

Program 

QT&E 

Qualification  Test  and  Eval¬ 

XOOST 

uation 

XORT 

OT&E  Division  (Air  Staff) 

ROM 

Requirements  Correlation 

XP 

Dir/DCS  of  Plans  and  PoU^ 

Matrix 

XR 

Dii/DCS  of  Plans,  Poli^, 

R&D 

Researdi  &  Development 

and  Requirements 

RDT&E 

Researdi,  Development,  Test 
and  Evaluation 

RM 

Director  of  Resource  Manage¬ 
ment  and  Support 

RTO 

Responsible  Test  Organization 

SA 

Director  of  Systems  Analysis 

SAC 

Strategic  Air  Command 

SAE 

Service  Acquisition  Executive 

SAP 

Secretary  of  the  Air  Force 

